{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Writing Low-Level TensorFlow Code\n",
    "\n",
    "\n",
    "**Learning Objectives**\n",
    "\n",
    " 1. Practice defining and performing basic operations on constant Tensors\n",
    " 2. Use Tensorflow's automatic differentiation capability\n",
    " 3. Learn how to train a linear regression from scratch with TensorFLow\n",
    "\n",
    "\n",
    "## Introduction \n",
    "\n",
    "In this notebook, we will start by reviewing the main operations on Tensors in TensorFlow and understand how to manipulate TensorFlow Variables. We explain how these are compatible with python built-in list and numpy arrays. \n",
    "\n",
    "Then we will jump to the problem of training a linear regression from scratch with gradient descent. The first order of business will be to understand how to compute the gradients of a function (the loss here) with respect to some of its arguments (the model weights here). The TensorFlow construct allowing us to do that is `tf.GradientTape`, which we will describe. \n",
    "\n",
    "At last we will create a simple training loop to learn the weights of a 1-dim linear regression using synthetic data generated from a linear model. \n",
    "\n",
    "As a bonus exercise, we will do the same for data generated from a non linear model, forcing us to manual engineer non-linear features to improve our linear model performance.\n",
    "\n",
    "Each learning objective will correspond to a #TODO in the [student lab notebook](https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive2/introduction_to_tensorflow/labs/write_low_level_code.ipynb) -- try to complete that notebook first before reviewing this solution notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here we'll import data processing libraries like Numpy and Tensorflow\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "# Use matplotlib for visualizing the model\n",
    "from matplotlib import pyplot as plt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "2.5.0\n"
      ]
     }
   ],
   "source": [
    "# Here we'll show the currently installed version of TensorFlow\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Operations on Tensors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variables and Constants"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensors in TensorFlow are either constant (`tf.constant`) or variables (`tf.Variable`).\n",
    "Constant values can not be changed, while variables values can be.\n",
    "\n",
    "The main difference is that instances of `tf.Variable` have methods allowing us to change \n",
    "their values while tensors constructed with `tf.constant` don't have these methods, and\n",
    "therefore their values can not be changed. When you want to change the value of a `tf.Variable`\n",
    "`x` use one of the following method: \n",
    "\n",
    "* `x.assign(new_value)`\n",
    "* `x.assign_add(value_to_be_added)`\n",
    "* `x.assign_sub(value_to_be_subtracted`\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(3,), dtype=int32, numpy=array([2, 3, 4], dtype=int32)>\n"
      ]
     }
   ],
   "source": [
    "# Creates a constant tensor from a tensor-like object.\n",
    "x = tf.constant([2, 3, 4])\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The Variable() constructor requires an initial value for the variable, which can be a Tensor of any type and shape.\n",
    "x = tf.Variable(2.0, dtype=tf.float32, name='my_variable')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Variable 'my_variable:0' shape=() dtype=float32, numpy=45.8>\n"
      ]
     }
   ],
   "source": [
    "# The .assign() method will assign the value to referance object.\n",
    "x.assign(45.8)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Variable 'my_variable:0' shape=() dtype=float32, numpy=49.8>\n"
      ]
     }
   ],
   "source": [
    "# The .assign_add() method will update the referance object by adding value to it.\n",
    "x.assign_add(4) \n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
    "name": "stdout",
    "output_type": "stream",
    "text": [
     "<tf.Variable 'my_variable:0' shape=() dtype=float32, numpy=46.8>\n"
    ]
   }
 ],
   "source": [
    "# The .assign_add() method will update the referance object by subtracting value to it.\n",
    "x.assign_sub(3) \n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Point-wise operations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Tensorflow offers similar point-wise tensor operations as numpy does:\n",
    "    \n",
    "* `tf.add` allows to add the components of a tensor \n",
    "* `tf.multiply` allows us to multiply the components of a tensor\n",
    "* `tf.subtract` allow us to substract the components of a tensor\n",
    "* `tf.math.*` contains the usual math operations to be applied on the components of a tensor\n",
    "* and many more...\n",
    "\n",
    "Most of the standard arithmetic operations (`tf.add`, `tf.substrac`, etc.) are overloaded by the usual corresponding arithmetic symbols (`+`, `-`, etc.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "# Here tf.Tensor() represents a multidimensional array of elements.\n",
       "c: tf.Tensor([ 8  2 10], shape=(3,), dtype=int32)\n",
       "d: tf.Tensor([ 8  2 10], shape=(3,), dtype=int32)\n"
      ]
     }
   ],
   "source": [
    "# Creates a constant tensor from a tensor-like object.\n",
    "a = tf.constant([5, 3, 8]) # TODO 1a\n",
    "b = tf.constant([3, -1, 2])\n",
    "# Using the .add() method components of a tensor will be added.\n",
    "c = tf.add(a, b)\n",
    "d = a + b\n",
    "\n",
    "# Let's output the value of `c` and `d`.\n",
    "print(\"c:\", c)\n",
    "print(\"d:\", d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "c: tf.Tensor([15 -3 16], shape=(3,), dtype=int32)\n",
       "d: tf.Tensor([15 -3 16], shape=(3,), dtype=int32)\n"
      ]
     }
   ],
   "source": [
    "# Creates a constant tensor from a tensor-like object.\n",
    "a = tf.constant([5, 3, 8]) # TODO 1b\n",
    "b = tf.constant([3, -1, 2])\n",
    "# Using the .multiply() method components of a tensor will be multiplied.\n",
    "c = tf.multiply(a, b)\n",
    "d = a * b\n",
    "\n",
    "# Let's output the value of `c` and `d`.\n",
    "print(\"c:\", c)\n",
    "print(\"d:\", d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "b: tf.Tensor([ 148.41316    20.085537 2980.958   ], shape=(3,), dtype=float32)\n"
      ]
     }
   ],
   "source": [
    "# TODO 1c\n",
    "# tf.math.exp expects floats so we need to explicitly give the type\n",
    "a = tf.constant([5, 3, 8], dtype=tf.float32)\n",
    "b = tf.math.exp(a)\n",
    "\n",
    "# Let's output the value of `b`.\n",
    "print(\"b:\", b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NumPy Interoperability\n",
    "\n",
    "In addition to native TF tensors, tensorflow operations can take native python types and NumPy arrays as operands. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# native python list\n",
    "a_py = [1, 2] \n",
    "b_py = [3, 4] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([4, 6], dtype=int32)>\n"
      ]
     }
   ],
   "source": [
    "# Using the .add() method components of a tensor will be added.\n",
    "tf.add(a_py, b_py)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# numpy arrays\n",
    "a_np = np.array([1, 2])\n",
    "b_np = np.array([3, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(2,), dtype=int64, numpy=array([4, 6])>\n"
      ]
     }
   ],
   "source": [
    "# Using the .add() method components of a tensor will be added.\n",
    "tf.add(a_np, b_np)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# native TF tensor\n",
    "a_tf = tf.constant([1, 2])\n",
    "b_tf = tf.constant([3, 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([4, 6], dtype=int32)>\n"
      ]
     }
   ],
   "source": [
    "# Using the .add() method components of a tensor will be added.\n",
    "tf.add(a_tf, b_tf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "array([1, 2], dtype=int32)\n"
      ]
     }
   ],
   "source": [
    "# Here using the .numpy() method we'll convert a `native TF tensor` to a `NumPy array`.\n",
    "a_tf.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Linear Regression\n",
    "\n",
    "Now let's use low level tensorflow operations to implement linear regression.\n",
    "\n",
    "Later in the course you'll see abstracted ways to do this using high level TensorFlow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Toy Dataset\n",
    "\n",
    "We'll model the following function:\n",
    "\n",
    "\\begin{equation}\n",
    "y= 2x + 10\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "X:[0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]\n",
       "Y:[10. 12. 14. 16. 18. 20. 22. 24. 26. 28.]\n"
      ]
     }
   ],
   "source": [
    "# Creates a constant tensor from a tensor-like object.\n",
    "X = tf.constant(range(10), dtype=tf.float32)\n",
    "Y = 2 * X + 10\n",
    "\n",
    "# Let's output the value of `X` and `Y`.\n",
    "print(\"X:{}\".format(X))\n",
    "print(\"Y:{}\".format(Y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also create a test dataset to evaluate our models:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "X_test:[10. 11. 12. 13. 14. 15. 16. 17. 18. 19.]\n",
       "Y_test:[30. 32. 34. 36. 38. 40. 42. 44. 46. 48.]\n"
      ]
     }
   ],
   "source": [
    "# Creates a constant tensor from a tensor-like object.\n",
    "X_test = tf.constant(range(10, 20), dtype=tf.float32)\n",
    "Y_test = 2 * X_test + 10\n",
    "\n",
    "# Let's output the value of `X_test` and `Y_test`.\n",
    "print(\"X_test:{}\".format(X_test))\n",
    "print(\"Y_test:{}\".format(Y_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Loss Function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The simplest model we can build is a model that for each value of x returns the sample mean of the training set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The numpy().mean() will compute the arithmetic mean or average of the given data (array elements) along the specified axis.\n",
    "y_mean = Y.numpy().mean()\n",
    "\n",
    "\n",
    "# Let's define predict_mean() function.\n",
    "def predict_mean(X):\n",
    "    y_hat = [y_mean] * len(X)\n",
    "    return y_hat\n",
    "\n",
    "Y_hat = predict_mean(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using mean squared error, our loss is:\n",
    "\\begin{equation}\n",
    "MSE = \\frac{1}{m}\\sum_{i=1}^{m}(\\hat{Y}_i-Y_i)^2\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this simple model the loss is then:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "33.0\n"
      ]
     }
   ],
   "source": [
    "# Let's evaluate the loss.\n",
    "errors = (Y_hat - Y)**2\n",
    "loss = tf.reduce_mean(errors)\n",
    "loss.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This value for the MSE loss above will give us a baseline to compare how a more complex model is doing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, if $\\hat{Y}$ represents the vector containing our model's predictions when we use a linear regression model\n",
    "\\begin{equation}\n",
    "\\hat{Y} = w_0X + w_1\n",
    "\\end{equation}\n",
    "\n"
  ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's define loss_mse() function which is taking arguments as coefficients of the model\n",
    "def loss_mse(X, Y, w0, w1):\n",
    "    Y_hat = w0 * X + w1\n",
    "    errors = (Y_hat - Y)**2\n",
    "    return tf.reduce_mean(errors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Gradient Function\n",
    "\n",
    "To use gradient descent we need to take the partial derivatives of the loss function with respect to each of the weights. We could manually compute the derivatives, but with Tensorflow's automatic differentiation capabilities we don't have to!\n",
    "\n",
    "During gradient descent we think of the loss as a function of the parameters $w_0$ and $w_1$. Thus, we want to compute the partial derivative with respect to these variables. \n",
    "\n",
    "For that we need to wrap our loss computation within the context of `tf.GradientTape` instance which will record gradient information:\n",
    "\n",
    "```python\n",
    "with tf.GradientTape() as tape:\n",
    "    loss = # computation \n",
    "```\n",
    "\n",
    "This will allow us to later compute the gradients of any tensor computed within the `tf.GradientTape` context with respect to instances of `tf.Variable`:\n",
    "\n",
    "```python\n",
    "gradients = tape.gradient(loss, [w0, w1])\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's define compute_gradients() procedure for computing the loss gradients with respect to the model weights:\n",
    "# TODO 2\n",
    "def compute_gradients(X, Y, w0, w1):\n",
    "    with tf.GradientTape() as tape:\n",
    "        loss = loss_mse(X, Y, w0, w1)\n",
    "    return tape.gradient(loss, [w0, w1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The Variable() constructor requires an initial value for the variable, which can be a Tensor of any type and shape.\n",
    "w0 = tf.Variable(0.0)\n",
    "w1 = tf.Variable(0.0)\n",
    "\n",
    "dw0, dw1 = compute_gradients(X, Y, w0, w1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "dw0 -204.0\n"
      ]
     }
   ],
   "source": [
    "# Let's output the value of `dw0`.\n",
    "print(\"dw0:\", dw0.numpy())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "dw1 -38.0\n"
      ]
     }
   ],
   "source": [
    "# Let's output the value of `dw1`.\n",
    "print(\"dw1\", dw1.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training Loop\n",
    "\n",
    "Here we have a very simple training loop that converges. Note we are ignoring best practices like batching, creating a separate test set, and random weight initialization for the sake of simplicity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "STEP 0 - loss: 35.70719528198242, w0: 4.079999923706055, w1: 0.7599999904632568\n",
       "STEP 100 - loss: 2.6017532348632812, w0: 2.4780430793762207, w1: 7.002389907836914\n",
       "STEP 200 - loss: 0.26831889152526855, w0: 2.153517961502075, w1: 9.037351608276367\n",
       "STEP 300 - loss: 0.027671903371810913, w0: 2.0493006706237793, w1: 9.690855979919434\n",
       "STEP 400 - loss: 0.0028539239428937435, w0: 2.0158326625823975, w1: 9.90071964263916\n",
       "STEP 500 - loss: 0.0002943490108009428, w0: 2.005084753036499, w1: 9.96811580657959\n",
       "STEP 600 - loss: 3.0356444767676294e-05, w0: 2.0016329288482666, w1: 9.989760398864746\n",
       "STEP 700 - loss: 3.1322738323069643e-06, w0: 2.0005245208740234, w1: 9.996710777282715\n",
       "STEP 800 - loss: 3.2238213520940917e-07, w0: 2.0001683235168457, w1: 9.998944282531738\n",
       "STEP 900 - loss: 3.369950718479231e-08, w0: 2.000054359436035, w1: 9.999658584594727\n",
       "STEP 1000 - loss: 3.6101481803996194e-09, w0: 2.0000178813934326, w1: 9.99988842010498\n"
      ]
     }
   ],
   "source": [
    "# TODO 3\n",
    "STEPS = 1000\n",
    "LEARNING_RATE = .02\n",
    "MSG = \"STEP {step} - loss: {loss}, w0: {w0}, w1: {w1}\\n\"\n",
    "\n",
    "\n",
    "# The Variable() constructor requires an initial value for the variable, which can be a Tensor of any type and shape.\n",
    "w0 = tf.Variable(0.0)\n",
    "w1 = tf.Variable(0.0)\n",
    "\n",
    "\n",
    "for step in range(0, STEPS + 1):\n",
    "\n",
    "    dw0, dw1 = compute_gradients(X, Y, w0, w1)\n",
    "    w0.assign_sub(dw0 * LEARNING_RATE)\n",
    "    w1.assign_sub(dw1 * LEARNING_RATE)\n",
    "\n",
    "    if step % 100 == 0:\n",
    "        loss = loss_mse(X, Y, w0, w1)\n",
    "        print(MSG.format(step=step, loss=loss, w0=w0.numpy(), w1=w1.numpy()))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
       "2.4563633e-08\n"
      ]
     }
   ],
   "source": [
    "# Here we can compare the test loss for this linear regression to the test loss from the baseline model.\n",
    "# Its output will always be the mean of the training set:\n",
    "loss = loss_mse(X_test, Y_test, w0, w1)\n",
    "loss.numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is indeed much better!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try modeling a non-linear function such as: $y=xe^{-x^2}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = tf.constant(np.linspace(0, 2, 1000), dtype=tf.float32)\n",
    "Y = X * tf.exp(-X**2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "data": {
       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoOklEQVR4nO3deXhU5d3/8fd3spJANkjYAgQIEMIaCIiIC2gVEIsrAipulVKlLq3bo7bPr9qnra37iqCoqEDdUAQUd6EgS1gSCGsIhKwkEJIQsif3748ETSPLJMzkzPJ9XReXycwZ5uNw+HByn/vcR4wxKKWUcn82qwMopZRyDC10pZTyEFroSinlIbTQlVLKQ2ihK6WUh/C16o07dOhgYmJirHp7pZRyS5s2bTpsjIk82XOWFXpMTAxJSUlWvb1SSrklEck41XM65KKUUh5CC10ppTyEFrpSSnkILXSllPIQWuhKKeUhtNCVUspDaKErpZSHsGweuvI8NbV1ZBeVk374OAUllRSXV3O8qgZfm+DrYyM8yI+OIYF0CWtDTPtg/H31eEIpR9JCVy1WU1vHhv2FrE47zMb9haRkF1NVU2fXa/19bPTr1I7B0aGc3yeS82Lb0y7Qz8mJlfJsWuiq2VKyili0IZOVqXkUHq/C1yYM7BrKjFE96NuxHb0ig+kYEkhokB/B/r7U1hmqa+soPF7FoZIKsovK2ZFbQmp2CZ9uzeG99QfxtQnn9m7PlUO7Mn5gJ4IDdNdUqrnEqjsWJSYmGr30333U1RlWbM9l3ur9JGcWEeTvwyX9OzJxUCcu6BtJkH/LCri6to7NGUf5fk8By1NyOVhYRhs/H65M6MrtY2KIjWrn4P8TpdybiGwyxiSe9DktdHU6xhi+2nGIZ77aw668Y/SKDGbGqB5cPTyaEAcPkRhj2JRxlA+SsvhkazaVNXWM7RfJPZf0ZWi3MIe+l1LuSgtdtUh6QSl/+nQ7a9KO0LNDMPde0odJg7vgYxOnv/eR0kreW3+Qt9YeoPB4FZcN6Mj9l/ajT0c9YlfeTQtdNUt1bR0vf5fGK9/tI8DPxoOX9WPayO74+rT+rJTSyhreWL2feavTKauq4aZRPfjDpf0IbaMnUJV30kJXdss4cpy7F20hOauYyUO78Ojl/YlqF2h1LAqPV/Hc13t4d10GEcEBPHZ5fyYP7YKI839aUMqVaKEru3y6NZtHPt6Gj034xzWDmTios9WRfmFbVjGPfbKN5KxixsVF8Y9rBrnEPzhKtZbTFbpe2aGorTP8/fOd3LN4K/FdQvj83gtcsswBBkWH8vGd5/HnSfGsSTvMZc+u4vNtuVbHUsolaKF7uWMV1cxckMRrP6Rz46juLLxjFF3D2lgd67R8bMJtY3qy/O4xRIcH8bv3NvPAB8mUV9VaHU0pS9lV6CIyXkR2i0iaiDx8mu1GiEitiFzruIjKWY6UVjJt3jq+31PAE5MH8NcrB+FnwYnPloqNasfHd45m9thYPtycxdWvruXA4eNWx1LKMmf82ysiPsDLwAQgHpgmIvGn2O5JYKWjQyrHyykq57rXfiQtv5TXb07kpnNjrI7UIn4+Nu6/rB/zbxlBbnE5V7z0H75MzbM6llKWsOdwbCSQZoxJN8ZUAYuBySfZ7vfAR0C+A/MpJzh4pIzr5vxIQUklC247h7H9oqyOdNbG9ovis9ljiGkfzMx3NvHyd2lYdcJfKavYU+hdgcxG32c1PPYTEekKXAXMcVw05Qw5ReVMm7eO41U1LJo5ipE9I6yO5DDdIoL4YNa5TB7ahX+t3M2DH6bYvViYUp7AngU4TjbRt+mhz3PAQ8aY2tPNCxaRmcBMgO7du9sZUTlK/rEKbnh9PSXl1Sy8YxQDu4ZaHcnhAv18eO76ocS0D+b5b/aSebSMOTcOJyzI3+poSjmdPUfoWUC3Rt9HAzlNtkkEFovIAeBa4BURubLpb2SMmWuMSTTGJEZGRrYssWqRorIqbnp9A3nFFbx12wgGRXtemZ8gItz3q748d/1QNmcUce2cH8ktLrc6llJOZ0+hbwT6iEhPEfEHpgJLG29gjOlpjIkxxsQAHwJ3GmM+cXRY1TKVNbXMXLCJ/YeP8/rNiQzv4TnDLKdzZUJXFtw+krziCq599Uf26wwY5eHOWOjGmBpgNvWzV3YC7xtjUkVklojMcnZAdXaMMTz80TY2HCjkX9cN5rzYDlZHalWjerVn8cxRlFfXct2ctaTmFFsdSSmn0Uv/PdxzX+/hua/3cv+lfZk9ro/VcSyTll/KjDfWc6yyhjdvGUFijHf8lKI8j17676U+S87hua/3cu3waO4aG2t1HEvFRrXlg9+NJrJtADPmb2DjgUKrIynlcFroHmrPoWM8+GEKiT3C+dtVg3RVQqBrWBsWzxxFp5BAbpm/gU0ZWurKs2ihe6BjFdXMemcTbQN9eeWGYfj76h/zCVEhgSyaOYqokEBunr+RTRlHrY6klMPo33QPY4zh/g+SySgs4+Xpw4gK0aVlm+oYEsiiO0YR2S6Am+dvYPNBLXXlGbTQPcwb/9nPytRD/M+EOI+6CtTROoXWl3qHtv7cPH+Dzn5RHkEL3YOk5hTzzy9286v4jtw+pqfVcVxep9BAFt4xinYBvtw8f6Ou1Kjcnha6hyivquWexVsJC/LjyWsG60lQO3UJa8OC28+hzhhufGM9ecUVVkdSqsW00D3E31bsJC2/lKenDCEiWNctaY7YqLa8desIjh6vYsb89RSVVVkdSakW0UL3AN/sPMQ76zL4zZienN9H18hpicHRYcy7OZEDh8u45c2NHK+ssTqSUs2mhe7misqqeOijbcR1ascD4/tZHcetje7dgRenJ5CSVcRdCzdTU6tL7yr3ooXu5h5ftoOisiqenjKEAF8fq+O4vcsGdOKvVw7i+90F/O/SVL1JhnIr9qyHrlzUd7vz+XhzNrPHxjKgi+cuh9vapp/TnYOFZcz5YR/dIoKYdWFvqyMpZRctdDd1rKKaRz/eRmxUW35/sXev0+IMD17Wj6yjZfzj811Eh7dh0uAuVkdS6oy00N3Uk1/sIrekgg9njdahFiew2YSnrhvCoZIK/vB+Mp1CAnWFRuXydAzdDW3KKOTddQe5dXRPhvcItzqOxwr082HuTYl0DWvDHQuS9AYZyuVpobuZmto6Hvsklc6hgfzx0r5Wx/F44cH+vHXrCAB+8/ZGSiqqLU6k1KlpobuZd9ZlsDO3hD9Piic4QEfMWkOP9sG8euNwMo6U8fuFW6it05kvyjVpobuR/GMVPPPlHs7v04HxAztZHcerjOrVnscnD+SHPQX84/OdVsdR6qT0EM+N/H3FLipr6nh88kBdq8UC08/pzu68Euat3k+/TiFcOzza6khK/Rc9QncT69OPsGRLNjMv6EXPDsFWx/Faf5oUz3mx7Xnk4216xyPlcrTQ3UBdneHxZTvoGtbG6+8NajVfHxsvTx9Gl7BAfvvOJrKLyq2OpNRPtNDdwMdbsknNKeHB8f1o469zzq0WFuTP6zcnUlldx8wFSVRU11odSSlAC93llVfV8tTK3QyJDuUKvVrRZcRGteP5aUNJzSnhkSXbdM0X5RK00F3cvNXp5JVU8NikeGw2PRHqSsbFdeTeS/rw8eZs3l2XYXUcpbTQXVl+SQVzftjHhIGdGKGXnbuku8f1YVxcFI8v26EnSZXltNBd2NNf7qG6to6HJ8RZHUWdgs0mPDtlKF3C2vC7dzeTf0xvYaeso4XuonbnHeP9TZnMODeGHu11mqIrCw3yY86NwympqGb2e1uo1htjKItoobuop7/cTVt/X2brNEW30L9zCE9eM5gNBwr52wq9klRZQwvdBSVnFvHljkPccUEvwvWGz25j8tCu3HpeDG+uOcCnW7OtjqO8kBa6C3rqy91EBPtz25ieVkdRzfTIxP6MjIngoY9S2JVXYnUc5WW00F3Mj/uOsHrvYe68qDdtdTVFt+PnY+OlGxJoF+jHne9uprSyxupIyotoobsQYwxPfbmbjiEB3Diqh9VxVAtFtQvkxWkJHDhynIc/StGLjlSr0UJ3Id/tzmdTxlHuvrgPgX56ib87G9WrPX+8tB/LUnL1oiPVarTQXYQxhqe/3EP3iCCmJHazOo5ygN9d2Jux/SJ5YtlOUrKKrI6jvIAWuov4dlc+qTkl/H5cLH4++sfiCWw24ZkpQ+nQ1p8739tMcZnevk45lzaHCzDG8MK3aUSHt+HKhK5Wx1EOFB7sz0s3DONQSQV//CBZx9OVU2mhu4DVew+TnFnEnRfp0bknGtY9nP+Z0J+vdx5i3up0q+MoD2ZXe4jIeBHZLSJpIvLwSZ6fLCIpIrJVRJJEZIzjo3omYwwvfruXzqGBXDNcj8491a3nxTBhYCee/GI3SQd0ES/lHGcsdBHxAV4GJgDxwDQRiW+y2TfAEGPMUOA24HUH5/RY69IL2XjgKLMu7E2Ar85s8VQiwpPXDiY6vA2zF27hSGml1ZGUB7LnCH0kkGaMSTfGVAGLgcmNNzDGlJqfBweDAR0otNML3+wlsl0A14/QmS2eLiTQj1duGEZhWRX3/nsrtXX610Q5lj2F3hXIbPR9VsNj/0VErhKRXcBy6o/S1RkkHSjkx/Qj/PaCXjrv3EsM6BLKX349gNV7D/Pq92lWx1Eexp5CP9ltcn5xaGGMWWKMiQOuBJ446W8kMrNhjD2poKCgWUE90UvfpRER7M/0c7pbHUW1oqkjuvHrIV145qs9bNiv4+nKcewp9Cyg8XhANJBzqo2NMauA3iLS4STPzTXGJBpjEiMjI5sd1pPszC3h+90F3HZeDEH+umaLNxER/u+qgXSPCOLuRVsoPF5ldSTlIewp9I1AHxHpKSL+wFRgaeMNRCRWRKTh62GAP3DE0WE9ybxV6QT5++iaLV6qXaAfL00fRuHxKu7/IJk6HU9XDnDGQjfG1ACzgZXATuB9Y0yqiMwSkVkNm10DbBeRrdTPiLne6BUUp5RTVM7S5ByuH9GNsCBd79xbDewayqOX9+fbXfm88Z/9VsdRHsCun/WNMSuAFU0em9Po6yeBJx0bzXO9uWY/Brhd1zv3ejPO7cHafYd58otdJMaEk9A93OpIyo3pZYmtrLi8moXrDzJpcGeiw4OsjqMsJiL885ohdAwJZPbCLbreizorWuitbOH6gxyvqmXmBb2sjqJcRGiQHy9OT+BQSQUP6frp6ixoobeiyppa3lyznzGxHRjQJdTqOMqFDOsezoPj+/FFah7v6PrpqoW00FvRp1tyyD9WyW8v1KNz9Uu/GdOLsf0i+euynWzPLrY6jnJDWuitpK7OMHd1OvGdQxgT+4sp+kphswlPTxlKeLAfsxfq/UhV82mht5JVewtIyy/ljgt60jBlX6lfiAj254WpCRwsLOORj7fpeLpqFi30VvLW2gNEtgvg8kFdrI6iXNw5vdpz3yV9WZqcw/tJmWd+gVINtNBbwb6CUr7fXcCN5/TA31c/cnVmd46N5bzY9vzv0lR25x2zOo5yE9ourWDB2gP4+9h0ES5lNx+b8Oz1Q2kb4MvshZspq9LxdHVmWuhOVlJRzYebspg0pDOR7QKsjqPcSFS7QJ67PoG0glL+39JUq+MoN6CF7mTvb8zkeFUtt47Wy/xV843p04G7Lorl/aQslmzJsjqOcnFa6E5UW2dY8GMGiT3CGRStFxKplrn3kj6MiAnn0SXbSS8otTqOcmFa6E707a58DhaWcet5enSuWs7Xx8YL0xII8LVx18ItVFTXWh1JuSgtdCd6c81+OocGctmAjlZHUW6uc2gbnp4yhJ25Jfzf8p1Wx1EuSgvdSXbnHWPtviPcdG4PfH30Y1Znb1xcR+44vyfvrMtgxbZcq+MoF6RN4yTvrsvA39fG1BE6VVE5zgOXxTGkWxgPfZjCwSNlVsdRLkYL3QmOV9awZEs2kwZ1JiJY70ikHMff18ZL0xJA4K6Fm6ms0fF09TMtdCf4ZGs2pZU13KD3C1VO0C0iiH9dO4Rt2cX8TcfTVSNa6A5mjOHddQfp3zmEYd3DrI6jPNT4gZ247byevP1jBstTdDxd1dNCd7AtmUXszC3hxlHddVVF5VQPT2gYT/8ohQOHj1sdR7kALXQHe3ddBsH+Pkwe2tXqKMrD+fvaeHl6Aj424a6Fm3V+utJCd6Sjx6tYlpLLVcO60jbA1+o4ygtEhwfx9HVDSM0p4a/Ld1gdR1lMC92BPtyURVVNHTfqyVDVii6J78jMC3rx7rqDfJacY3UcZSEtdAepqzO8t75+3Za4TiFWx1Fe5oHL+jG8RzgPf5Si6714MS10B1mz7zAHjpTp0bmyhJ+PjRenJeDva+PO93Q83VtpoTvIe+sOEh7kx/iBnayOorxUl7A2PHP9UHblHeMvn+n66d5IC90B8o9V8PXOQ1yX2I1APx+r4ygvNrZfFL+7qDeLNmTyyZZsq+OoVqaF7gAfb86mps5w/YhuVkdRij/+qi8jYyJ4ZMk20vJ1PN2baKGfJWMM/96YyYiYcHpHtrU6jlI/rZ/exs+Hu97bTHmVjqd7Cy30s7RhfyH7Dx/nel1VUbmQTqGBPHv9UPbkH+NPn263Oo5qJVroZ+nfSZm0C/Bl4iA9GapcywV9I5k9NpYPN2WxeMNBq+OoVqCFfhaKy6tZsS2XK4Z2IchfrwxVrufeS/pyfp8O/PnTVJIzi6yOo5xMC/0sLE3OoaK6jql6MlS5KB+b8PzUBCLbBXDne5spPF5ldSTlRFroZ+H9jZn07xzCoK6hVkdR6pQigv159cZhFByr5J7FW6itM1ZHUk6ihd5C27OL2ZZdzNQR3XSZXOXyBkeH8fjkAazee5jnvt5jdRzlJFroLfR+Uib+vjau1GVylZuYOrI71yd248Vv0/h6xyGr4ygn0EJvgYrqWj7Zks2EgZ0IDfKzOo5SdvvL5AEM6hrKfe9v1ZtieCAt9Bb4YnseJRU1emWocjuBfj68csMwfGzCrHc36UVHHsauQheR8SKyW0TSROThkzx/g4ikNPxaKyJDHB/Vdfx7YybdI4IY1bO91VGUarZuEUE8PzWB3YeO8ciSbRijJ0k9xRkLXUR8gJeBCUA8ME1E4ptsth+40BgzGHgCmOvooK4is7CMH9OPcO3waGw2PRmq3NOFfSO575K+LNmSzYIfM6yOoxzEniP0kUCaMSbdGFMFLAYmN97AGLPWGHO04dt1QLRjY7qOJQ0r2F2VoCdDlXubPTaWi+OieGLZDtanH7E6jnIAewq9K5DZ6PushsdO5Xbg85M9ISIzRSRJRJIKCgrsT+kijDF8vDmLc3u1p1tEkNVxlDorNpvw7NShdG8fxJ3vbSbraJnVkdRZsqfQTzaucNJBNxEZS32hP3Sy540xc40xicaYxMjISPtTuohNGUc5cKSMa4Z77A8gysuEBPoxb0YiVTV1zFywibKqGqsjqbNgT6FnAY2nc0QDv7gTrYgMBl4HJhtjPPLntw83ZRHk78MEvSuR8iC9I9vywrQEduaV8MCHKXqS1I3ZU+gbgT4i0lNE/IGpwNLGG4hId+Bj4CZjjEdehlZRXcvylFzGD+xEcIAuxKU8y9i4KB4aH8fylFxe+X6f1XFUC52xmYwxNSIyG1gJ+ADzjTGpIjKr4fk5wJ+B9sArDZfB1xhjEp0Xu/WtTM3jWGUN1w7T4RblmX57QS925JTw1Je76dexHZfEd7Q6kmomserHq8TERJOUlGTJe7fEjPkb2JdfyuoHx+p0ReWxyqtque61tRw4XMYnd40mNqqd1ZFUEyKy6VQHzHqlqB3yiiv4z94Crh7WVctcebQ2/j7MvSmRQD8bdyzYRHFZtdWRVDNoodthyZZs6gxcrcMtygt0CWvDqzcOJ+toGbMXbaamts7qSMpOWuhnYIzho81ZDO8RTs8OwVbHUapVjIiJ4InJA1m99zB/+WyHznxxE1roZ5CSVUxafinX6NG58jJTR3bntxf04p11Gby19oDVcZQddP7dGXy0OYsAXxuXD+5sdRSlWt1D4+PYf/g4TyzbQfeIIC7urzNfXJkeoZ9GZU0tn27N4dIBnQhto+ueK+9jswnPTR3KgC6h/H7RFnbklFgdSZ2GFvppfLergOLyaq4epgtxKe8V5O/L6zcnEtrGj9vf3sihkgqrI6lT0EI/jU+2ZNOhrT/nx3awOopSluoYEsgbN4+gpLya37ydpGu+uCgt9FMoLq/m2135TBrcBV8f/ZiUiu8SwovTE0jNKebexVupq9OZL65Gm+oUvtieS1VtHVfquudK/WRcXEf+NCmeL3cc4vFlOp3R1egsl1P4ZEsOMe2DGBIdanUUpVzKLaNjyCwsZ/6a/XQODeS3F/a2OpJqoIV+ErnF5azbf4R7Lu5Dw2JjSqkGIsJjl/cn/1gFf/98F1EhAVyVoNdpuAIt9JNYujUHY+DKoTrcotTJ2GzC01OGcKS0igc+SKFD2wDO7+N+N63xNDqGfhKfbM1hSLcwYvRSf6VOKcDXh9dmDCc2qi2z3tnE9uxiqyN5PS30JnbnHWNnbglXDe1idRSlXF5IoB9v3zaSsCB/bnlzIweP6H1JraSF3sQnW7PxsQmThmihK2WPjiGBvH3bCKpr67j5zQ0cKa20OpLX0kJvpK7OsHRrDmNiO9ChbYDVcZRyG7FR7Zh/SyI5ReXMmL+BkgpdR90KWuiNJGUcJbuonCsT9OhcqeYa3iOCOTcOZ3feMW5/ayPlVbVWR/I6WuiNLNmSTRs/Hy6N72R1FKXc0ti4KJ6fmsCmjKPMfCeJyhot9dakhd6gqqaOFdtyuXRAR4IDdDanUi11+eDO/OPqwazee5h7Fm3VOx61Ii30Bt/vzqe4vFrnnivlAFNGdONPk+L5IjWPhz7apuu+tBI9FG3w6dYcIoL9GdNHV1ZUyhFuH9OT0ooanv16D+0CffnfK+L1ymsn00IHSiqq+WrnIaaN6IafrqyolMPcfXEspZXVzFu9H39fG/8zIU5L3Ym00IGV2/Ooqqljsq6sqJRDiQiPTOxPZU0dc1elA2ipO5EWOvBZSi7R4W1I6BZmdRSlPI6I8JdfDwBg7qp0BHhYS90pvL7Qj5RWsibtMDMv6KU7mFJOcqLUjYHXVqWDwMPjtdQdzesL/YvUPGrrDFcM1ouJlHImEeHxyQMwGF77oX74RUvdsby+0D9LzqF3ZDD9O7ezOopSHk9EeGLyQABe+yEdY3RM3ZG8utAPlVSwfn+h3shCqVYkIjz+64EIwtxV6RyvrOGJyQOx2fTv4Nny6kJfnpKLMTBJh1uUalU2W/3wS1CAD6/9kE5ZVS3/unaw3pD9LHl1oX+WkkN85xBio9paHUUpryMiPDw+jpBAP/61cjdlVTW8MC2BAF8fq6O5La/95zCzsIwtB4u4Qtc9V8oyIsJdY2P5f1fEszL1EL95O4myqhqrY7ktry30ZSm5AEwa3NniJEqpW87ryT+vHcyatMPMeGMDxWW6nnpLeG2hf5acQ0L3MLpFBFkdRSkFTEnsxovThpGcVcR1r60lp6jc6khuxysLPS2/lB25JTr3XCkXc/ngzrx960hyiyq46pU17MwtsTqSW/HKQl+WkoNI/c6jlHIto2M78P6scwGYMudH1u47bHEi9+F1hW6M4bPkHM7pGUHHkECr4yilTqJ/5xA+vvM8OoUGcvP8DSxNzrE6kluwq9BFZLyI7BaRNBF5+CTPx4nIjyJSKSL3Oz6m4+zMPca+guM691wpF9c1rA0fzhpNQrdw7l60hTk/7MMYvVHG6Zyx0EXEB3gZmADEA9NEJL7JZoXA3cBTDk/oYJ+l5OBjEyYM1PuGKuXqQoP8WHD7yPrb2n2+i/s/SNH7lJ6GPUfoI4E0Y0y6MaYKWAxMbryBMSbfGLMRcOm5RieGW86L7UD7tgFWx1FK2SHQz4cXpyZwz8V9+GhzFjfMW8/h0kqrY7kkewq9K5DZ6PushseaTURmikiSiCQVFBS05Lc4K1szi8g6Ws4VejJUKbdiswn3/aovL01PYFt2MZNfWsOuPJ0B05Q9hX6yFXNaNJBljJlrjEk0xiRGRka25Lc4K58l5+LvY+PSATrcopQ7mjS4Cx/MOpeaujqueWUtX+04ZHUkl2JPoWcB3Rp9Hw243Snn2jrDspQcLuwXSWgbP6vjKKVaaHB0GJ/eNYbeUW25Y0EST63cTW2dniwF+wp9I9BHRHqKiD8wFVjq3FiOt/FAIfnHKnXtFqU8QKfQQN7/7blMSYzmpe/SuOXNDRQer7I6luXOWOjGmBpgNrAS2Am8b4xJFZFZIjILQEQ6iUgW8AfgMRHJEpEQZwZvrs+Scwj0s3FJ/yiroyilHCDQz4d/XjuEf1w9iPX7C7nixf+QnFlkdSxL2TUP3RizwhjT1xjT2xjzfw2PzTHGzGn4Os8YE22MCTHGhDV87TJnLGrrDCtT87g4riNB/l69YrBSHmfqyO582HBl6XVzfuS99RleO1/dK64UXb//CIdLq/RSf6U81ODoMJb9fgyjerfn0SXbmb1wC8XlLj2L2im8otCXp+TSxs+Hsf10uEUpTxUe7M9bt4zgofFxrEzNY+Lzq0k6UGh1rFbl8YV+YrhlXFwUbfz1TihKeTKbTfjdRb358Hej8bEJU177kRe+2es1s2A8vtBPDLdMHKTDLUp5i6Hdwlh+9xh+PaQLz3y1h2nz1pFZWGZ1LKfz+EJfsS2XQD8bY+Na/0ImpZR12gX68dzUBJ6ZMoQdOSWMf26Vx58w9ehCr60zfLFdZ7co5c2uHhbNF/eez9DuYTy6ZDsz5m/w2LsheXSh63CLUgogOjyId28/hyeuHMimjKNc9uwq3t+Y6XFH6x5d6DrcopQ6QUS4aVQPvrjnAuK7hPDgRync+MZ69h8+bnU0h/HYQq8fbjnEuLgoHW5RSv2ke/sgFt0xiicmDyAls5jLnlvFC9/s9Yh11j220DfsL+RwaSWXD9K1W5RS/81mE246N4Zv/nghl8Z35Jmv9jDh+dWsSz9idbSz4rGFvnxbjg63KKVOKyokkJemD+OtW0dQXVvH1LnruO/fW8krrrA6Wot4ZKHrcItSqjku6hfFl/deyOyxsSzflsvYp77n+a/3Ul7lXsMwHlnoJ4ZbdHaLUspebfx9uP+yfnzzhwsZGxfJs1/v4eKnv2dpco7bzIbxyEI/MbtlXJyu3aKUap5uEUG8csNwFs8cRViQP3cv2sLVr65l7b7DVkc7I48r9No6w+fb83S4RSl1Vkb1as9nvx/Dk9cMIreogunz1nPTG+tJySqyOtopeVyh63CLUspRfGzC9SO68/0DF/HY5f3Znl3Mr19aw6x3NpGWf8zqeL/gcYewOtyilHK0QD8ffnN+L64f0Y3XV+/n9dXpfLkjj4mDOnPX2Fj6d3aNG7R5VKGfGG4Z20+HW5RSjtcu0I/7ftWXm0fHMHdVOu+uy2BZSi4Xx0Vx17hYhnUPtzSfRw25bDygwy1KKeeLCPbn4QlxrHloHH/4VV82HTzK1a+sZfq8dazaU2DZrBiPKvTlKTrcopRqPaFBftx9cR/WPDSOxy7vT1p+KTPmb+BXz9Yv1dva89g9ptAbD7cEB+hwi1Kq9QQH+PKb83ux+qGxPDNlCIF+Nh5dsp1Rf/+Gv3++k+xWWq7XY5pPh1uUUlYL8PXh6mHRXJXQlaSMo7y5Zj/zVqUzb1U64+I6Mv2cblzYNwofmzjl/T2m0FdsyyXAV4dblFLWExFGxEQwIiaCrKNlvLf+IB8kZfL1zkN0DWvDPZf0YUpiN4e/r0cUeuOLiXS4RSnlSqLDg3hofBz3XdKXr3ceYtGGg067abVHtN/GA4UUHNPhFqWU6/L3tTFxUGcmDurstFkwHnFSVIdblFLuRMQ5Y+huX+g6u0Uppeq5faEnnRhuGazDLUop7+b2hb68YbjlYh1uUUp5ObcudB1uUUqpn7l1oetwi1JK/cytC32FDrcopdRP3LbQTwy3XNQvUodblFIKNy70pAOF5B+r5PLBXayOopRSLsFtC12HW5RS6r+5ZaHX6XCLUkr9glsWelLGUfJ17RallPovdhW6iIwXkd0ikiYiD5/keRGRFxqeTxGRYY6P+rPlKTn1wy39OzrzbZRSyq2csdBFxAd4GZgAxAPTRCS+yWYTgD4Nv2YCrzo4508aD7e01eEWpZT6iT1H6COBNGNMujGmClgMTG6yzWRggam3DggTEaeMh+hwi1JKnZw9hd4VyGz0fVbDY83dBhGZKSJJIpJUUFDQ3KwA2AQu7Bupwy1KKdWEPYV+soV7m67Obs82GGPmGmMSjTGJkZGR9uT7hcSYCN6+baQOtyilVBP2FHoW0Pjmd9FATgu2UUop5UT2FPpGoI+I9BQRf2AqsLTJNkuBGQ2zXUYBxcaYXAdnVUopdRpnHLcwxtSIyGxgJeADzDfGpIrIrIbn5wArgIlAGlAG3Oq8yEoppU7GroFoY8wK6ku78WNzGn1tgLscG00ppVRzuOWVokoppX5JC10ppTyEFrpSSnkILXSllPIQUn8+04I3FikAMlr48g7AYQfGcRRXzQWum01zNY/mah5PzNXDGHPSKzMtK/SzISJJxphEq3M05aq5wHWzaa7m0VzN4225dMhFKaU8hBa6Ukp5CHct9LlWBzgFV80FrptNczWP5moer8rllmPoSimlfsldj9CVUko1oYWulFIewuUK/WxuSH2m1zo51w0NeVJEZK2IDGn03AER2SYiW0UkqZVzXSQixQ3vvVVE/mzva52c64FGmbaLSK2IRDQ858zPa76I5IvI9lM8b9X+daZcVu1fZ8pl1f51plytvn+JSDcR+U5EdopIqojcc5JtnLt/GWNc5hf1y/PuA3oB/kAyEN9km4nA59TfJWkUsN7e1zo512ggvOHrCSdyNXx/AOhg0ed1EbCsJa91Zq4m218BfOvsz6vh974AGAZsP8Xzrb5/2Zmr1fcvO3O1+v5lTy4r9i+gMzCs4et2wJ7W7i9XO0I/mxtS2/Nap+Uyxqw1xhxt+HYd9Xdtcraz+X+29PNqYhqwyEHvfVrGmFVA4Wk2sWL/OmMui/Yvez6vU7H082qiVfYvY0yuMWZzw9fHgJ388t7KTt2/XK3Qz+aG1HbdqNqJuRq7nfp/hU8wwJcisklEZjooU3NynSsiySLyuYgMaOZrnZkLEQkCxgMfNXrYWZ+XPazYv5qrtfYve7X2/mU3q/YvEYkBEoD1TZ5y6v7landaPpsbUtt1o+oWsvv3FpGx1P+FG9Po4fOMMTkiEgV8JSK7Go4wWiPXZurXfigVkYnAJ0AfO1/rzFwnXAGsMcY0Ptpy1udlDyv2L7u18v5lDyv2r+Zo9f1LRNpS/w/IvcaYkqZPn+QlDtu/XO0I/WxuSO3MG1Xb9XuLyGDgdWCyMebIiceNMTkN/80HllD/41Wr5DLGlBhjShu+XgH4iUgHe17rzFyNTKXJj8NO/LzsYcX+ZRcL9q8zsmj/ao5W3b9ExI/6Mn/PGPPxSTZx7v7l6BMDZ/OL+p8Y0oGe/HxiYECTbS7nv08qbLD3tU7O1Z36e6qObvJ4MNCu0ddrgfGtmKsTP19ANhI42PDZWfp5NWwXSv04aHBrfF6N3iOGU5/ka/X9y85crb5/2Zmr1fcve3JZsX81/H8vAJ47zTZO3b8c9uE68A9pIvVnh/cBjzY8NguY1ehDe7nh+W1A4ule24q5XgeOAlsbfiU1PN6r4Q8nGUi1INfshvdNpv5k2ujTvba1cjV8fwuwuMnrnP15LQJygWrqj4pud5H960y5rNq/zpTLqv3rtLms2L+oHwYzQEqjP6eJrbl/6aX/SinlIVxtDF0ppVQLaaErpZSH0EJXSikPoYWulFIeQgtdKaU8hBa6Ukp5CC10pZTyEP8fC7OtFlrn9uQAAAAASUVORK5CYII=\n",
       "text/plain": [
        "<AxesSubplot:>"
       ]
      },
      "metadata": {
       "needs_background": "light"
      },
      "output_type": "display_data"
     }
   ],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# The .plot() is a versatile function, and will take an arbitrary number of arguments. For example, to plot x versus y.\n",
    "plt.plot(X, Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's make_features() procedure.\n",
    "def make_features(X):\n",
    "# The tf.ones_like() method will create a tensor of all ones that has the same shape as the input.\n",
    "    f1 = tf.ones_like(X)\n",
    "    f2 = X\n",
    "# The tf.square() method will compute square of input tensor element-wise.\n",
    "    f3 = tf.square(X)\n",
    "# The tf.sqrt() method will compute element-wise square root of the input tensor.\n",
    "    f4 = tf.sqrt(X)\n",
    "# The tf.exp() method will compute exponential of input tensor element-wise.\n",
    "    f5 = tf.exp(X)\n",
    "# The tf.stack() method will stacks a list of rank-R tensors into one rank-(R+1) tensor.\n",
    "    return tf.stack([f1, f2, f3, f4, f5], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's define predict() procedure that will remove dimensions of size 1 from the shape of a tensor.\n",
    "def predict(X, W):\n",
    "    return tf.squeeze(X @ W, -1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's define loss_mse() procedure that will evaluate the mean of elements across dimensions of a tensor.\n",
    "def loss_mse(X, Y, W):\n",
    "    Y_hat = predict(X, W)\n",
    "    errors = (Y_hat - Y)**2\n",
    "    return tf.reduce_mean(errors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's define compute_gradients() procedure for computing the loss gradients.\n",
    "def compute_gradients(X, Y, W):\n",
    "    with tf.GradientTape() as tape:\n",
    "        loss = loss_mse(Xf, Y, W)\n",
    "    return tape.gradient(loss, W)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "data": {
       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD6CAYAAACoCZCsAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAAjUUlEQVR4nO3deXhc1X3G8e9Po30dWZJlW5ItG2wwYMvYigVhK8EkLAkmS1NIWgdI6jgJBZImKWnShjZNm6ZZGgrFNQlNoCQQCElc4oQAZQmbsQ3G+yJvWLIsW7K179LpH3PtjIWWkTXSlTTv53nmmZl7z9H85kqad+49dzHnHCIiEnvi/C5ARET8oQAQEYlRCgARkRilABARiVEKABGRGKUAEBGJUREFgJldZWY7zazczO7sY76Z2d3e/E1mtnCwvma2wMxeM7ONZrbezBZH5y2JiEgkbLDjAMwsAOwCrgQqgHXAjc65bWFtrgH+CrgGKAN+4JwrG6ivmf0e+L5z7rde/y875/5koFpyc3NdcXHxab1REZFYtWHDhhrnXF7v6fER9F0MlDvn9gKY2SPAUmBbWJulwIMulCavmVnQzKYCxQP0dUCm1z8LODRYIcXFxaxfvz6CkkVE5AQzO9DX9EgCoAA4GPa8gtC3/MHaFAzS9w7gKTP7DqFNUe/up/DlwHKA6dOnR1CuiIhEIpIxAOtjWu/tRv21GajvZ4DPO+eKgM8DP+rrxZ1zq5xzpc650ry8d6zBiIjIaYokACqAorDnhbxzc01/bQbq+wngCe/xY4Q2NYmIyCiJJADWAbPNbKaZJQI3AKt7tVkNLPP2BroAqHfOVQ3S9xBwmff4PcDuYb4XEREZgkHHAJxzXWZ2K/AUEAAecM5tNbMV3vyVwBpCewCVAy3AzQP19X70XwI/MLN4oA1vO7+IiIyOQXcDHUtKS0ud9gISERkaM9vgnCvtPV1HAouIxKiYCIDndhzhP58v97sMEZExJSYC4JU9Nfz7M7vp6OrxuxQRkTEjJgKgpChIR1cPOw83+l2KiMiYERsBUBgEYGNFna91iIiMJTERAIXZKeSkJfLWwTq/SxERGTNiIgDMjJKioAJARCRMTAQAhDYDlR9torGt0+9SRETGhJgJgAXTgzgHmyvr/S5FRGRMiJkAKCnMAmCjNgOJiAAxFADB1ESKc1I1DiAi4omZAAC8gWBtAhIRgVgLgMIghxvaOFzf5ncpIiK+i60AKAoC8JYOCBMRia0AOHdaJvFxpnEAERFiLACSEwLMnZqpPYFERIixAAAoKcpiU0U9PT3j50I4IiIjIfYCoDBIU3sXe2ua/C5FRMRXMRcAC7yB4I3aHVREYlzMBcCsvHTSk+I1ECwiMS/mAiAQZ8wryNKuoCIS82IuACB0YrjtVQ20dXb7XYqIiG9iMgBKCoN0dju2VTX4XYqIiG9iMgBODARrHEBEYllMBsCUrGTyM5MUACIS02IyACC0GeitCu0KKiKxK3YDoCjIvppm6lo6/C5FRMQXMRsA53vjAJu0FiAiMSqiADCzq8xsp5mVm9mdfcw3M7vbm7/JzBYO1tfMHjWzjd5tv5ltjMo7itB5hVmY6RKRIhK74gdrYGYB4F7gSqACWGdmq51z28KaXQ3M9m5lwH1A2UB9nXN/FvYa3wVG9at4ZnICZ+SlayBYRGJWJGsAi4Fy59xe51wH8AiwtFebpcCDLuQ1IGhmUyPpa2YGfBT42TDfy5CFBoLrcE5nBhWR2BNJABQAB8OeV3jTImkTSd9LgGrn3O6+XtzMlpvZejNbf/To0QjKjdyCoixqmjqorGuN6s8VERkPIgkA62Na76/M/bWJpO+NDPDt3zm3yjlX6pwrzcvLG7DQoTp5iUidGVREYlAkAVABFIU9LwQORdhmwL5mFg98CHg08pKj5+wpmSTGx+nEcCISkyIJgHXAbDObaWaJwA3A6l5tVgPLvL2BLgDqnXNVEfRdAuxwzlUM+52chsT4OM6dpktEikhsGnQvIOdcl5ndCjwFBIAHnHNbzWyFN38lsAa4BigHWoCbB+ob9uNvwIfB33AlhUEeXXeQru4e4gMxe1iEiMSgQQMAwDm3htCHfPi0lWGPHfC5SPuGzbsp0kJHyoKiID9+ZT+7jzQxd2qm3+WIiIyamP/KW6Izg4pIjIr5ACjOSSUzOV4DwSISc2I+AMyMkqKgLhIvIjEn5gMAQuMAu6obaeno8rsUEZFRowAgFADdPY4tlbpEpIjEDgUAML8wCGggWERiiwIAyMtIoiCYwkYNBItIDFEAeBYUBbUGICIxRQHgKSnKouJ4KzVN7X6XIiIyKhQAnhJvHGCTNgOJSIxQAHjmFWYRZ+h4ABGJGQoAT2piPHPyM3RmUBGJGQqAMCcGgnWJSBGJBQqAMCVFQepbOzlQ2+J3KSIiI04BEObEQLBODCcisUABEGZOfjrJCXEaBxCRmKAACBMfiGNeQZYOCBORmKAA6GVBUZAthxro7O7xuxQRkRGlAOilpChIR1cPO6oa/S5FRGREKQB6OTEQrBPDichEpwDopTA7hZy0RI0DiMiEpwDo5cQlIhUAIjLRKQD6UFIYpPxoE41tnX6XIiIyYhQAfVgwPYhzsLlSJ4YTkYlLAdCHksIsAB0QJiITmgKgD8HURIpzUjUOICITmgKgH6GBYG0CEpGJK6IAMLOrzGynmZWb2Z19zDczu9ubv8nMFkbS18z+ypu31cy+Pfy3Ez0lhUEON7RxuL7N71JEREZE/GANzCwA3AtcCVQA68xstXNuW1izq4HZ3q0MuA8oG6ivmV0OLAXmO+fazWxyNN/YcJUUBYHQmUGnZE3xtxgRkREQyRrAYqDcObfXOdcBPELogzvcUuBBF/IaEDSzqYP0/QzwLedcO4Bz7kgU3k/UnDstk/g40ziAiExYkQRAAXAw7HmFNy2SNgP1nQNcYmZrzewFM3tXXy9uZsvNbL2ZrT969GgE5UZHckKAuVMzdW0AEZmwIgkA62Na72sm9tdmoL7xQDZwAfAl4Odm9o72zrlVzrlS51xpXl5eBOVGT0lRFpsO1tPTo0tEisjEE0kAVABFYc8LgUMRthmobwXwhLfZ6HWgB8iNvPSRV1IYpLG9i701TX6XIiISdZEEwDpgtpnNNLNE4AZgda82q4Fl3t5AFwD1zrmqQfr+CngPgJnNARKBmuG+oWha4A0Eb9TuoCIyAQ0aAM65LuBW4ClgO/Bz59xWM1thZiu8ZmuAvUA5cD/w2YH6en0eAGaZ2RZCg8OfcM6NqW0ts/LSSU+K10CwiExIg+4GCuCcW0PoQz582sqwxw74XKR9vekdwJ8PpdjRFoiz0CUiNRAsIhOQjgQexILpQbZXNdDW2e13KSIiUaUAGERJYZDObse2qga/SxERiSoFwCBODARrHEBEJhoFwCCmZCWTn5mkABCRCUcBEIGSwiAb3j6uA8JEZEJRAETg6nlTOHislcc3VPhdiohI1CgAIrC0pIDSGdn8y2+3c7y5w+9yRESiQgEQgbg44xvXn0dDWxfffmqn3+WIiESFAiBCc6dmctO7i3lk3du6VrCITAgKgCG4Y8ls8tKT+NqvNtOtAWERGecUAEOQkZzA373/HLZUNvDw2gN+lyMiMiwKgCF6//ypXHxmLv/21E6ONrb7XY6IyGlTAAyRmfEPS8+lrbObf/ntdr/LERE5bQqA03BGXjrLL53FE29UsnZvrd/liIicFgXAabr18tkUBFP4u19vobO7x+9yRESGTAFwmlISA9x13bnsqm7ixy/v97scEZEhUwAMw5Xn5HPF2ZP5/jO7qKpv9bscEZEhUQAM013XnUt3j+OfntSAsIiMLwqAYSqalMqtl5/JbzZX8eKuo36XIyISMQVAFCy/bBYzc9P4+uqttHfp0pEiMj4oAKIgKT7AP1x3Lvtqmln1wl6/yxERiYgCIEounZPHtfOmcs9z5Rw81uJ3OSIig1IARNHX3j+XQJxx1+qtfpciIjIoBUAUTc1K4fNL5vDsjiM8va3a73JERAakAIiymy4qZk5+Onet3kprhwaERWTsUgBEWUIgjm8sPY/KulbueW633+WIiPRLATACymbl8KGFBax6cS97jjb5XY6ISJ8UACPkK1fPJTkhwN//egvO6ephIjL2RBQAZnaVme00s3Izu7OP+WZmd3vzN5nZwsH6mtldZlZpZhu92zXReUtjQ15GEl9631m8XF7Lk5uq/C5HROQdBg0AMwsA9wJXA+cAN5rZOb2aXQ3M9m7Lgfsi7Pt959wC77ZmuG9mrPl42QzOK8jkG09uo7Gt0+9yREROEckawGKg3Dm31znXATwCLO3VZinwoAt5DQia2dQI+05YgTjjn66fx9Gmdv79GQ0Ii8jYEkkAFAAHw55XeNMiaTNY31u9TUYPmFl2Xy9uZsvNbL2ZrT96dPydbG1BUZAbF0/ngZf38X87dGyAiIwdkQSA9TGt96hmf20G6nsfcAawAKgCvtvXizvnVjnnSp1zpXl5eRGUO/Z87dq5nDM1k9t/tpHyI41+lyMiAkQWABVAUdjzQuBQhG367eucq3bOdTvneoD7CW0umpBSE+NZtayUpIQ4PvWT9dS3aDxARPwXSQCsA2ab2UwzSwRuAFb3arMaWObtDXQBUO+cqxqorzdGcMIHgS3DfC9jWkEwhZV/vojKulZu/dkbdOk6wiLis0EDwDnXBdwKPAVsB37unNtqZivMbIXXbA2wFygn9G3+swP19fp828w2m9km4HLg89F7W2NTafEkvnn9PP6wu4Z/XrPD73JEJMbZeDpIqbS01K1fv97vMobtrtVb+fEr+/n2R+bz0dKiwTuIiAyDmW1wzpX2nq4jgX3wtWvncvGZuXztl1vYcOCY3+WISIxSAPggPhDHPR87n6nBZD790Bscqmv1uyQRiUEKAJ8EUxP54bJS2jq7Wf7Qep06WkRGnQLAR7PzM/jBDQvYeqiBL/9ik04aJyKjSgHgsyvm5vOl953F/751iP98fo/f5YhIDFEAjAGfuewMriuZxnd+v5NndClJERklCoAxwMz49kfmc960LG5/5E12Vet0ESIy8hQAY0RyQoBVyxaRkhjPp36ynuPNHX6XJCITnAJgDJmalcJ//cUiDte38bmfvkGnThchIiNIATDGLJqRzT9/aB6v7Knlm7/Z7nc5IjKBxftdgLzTRxYVsqOqgR++tI+zp2Rww+LpfpckIhOQ1gDGqDuvPptLZufyd7/ewrr9Ol2EiESfAmCMig/Ecc+NCynMTmXFQxuo1OkiRCTKFABjWFZqAvcvK6Wjq4dP/nid9gwSkahSAIxxZ05O596PL2RvTTM3rHqNI41tfpckIhOEAmAcuHROHv9907s4eLyFj658VZuDRCQqFADjxEVn5vLQJxdT29zBR1e+yr6aZr9LEpFxTgEwjiyaMYmf/eUFtHZ289H/epWdh3XKCBE5fQqAcea8giweXX4BBvzZqlfZXFHvd0kiMk4pAMah2fkZPLbiQtIS4/nY/a/pOAEROS0KgHFqRk4aj624kLyMJJb96HVe2l3jd0kiMs4oAMaxacEUHv30hczISeWWH6/jaV1LQESGQAEwzuVlJPHI8guYOy2TFf+zgV9vrPS7JBEZJxQAE0AwNZGHP1XGohnZ3PHoRh55/W2/SxKRcUABMEGkJ8Xzk5sXc+nsPO58YjM/emmf3yWJyBinAJhAUhJDVxV737n5fOPJbdzzf7txzvldloiMUQqACSYpPsC9H1vIB88v4Du/38W//m6nQkBE+hRRAJjZVWa208zKzezOPuabmd3tzd9kZguH0PeLZubMLHd4b0VOiA/E8d0/LeFjZdNZ+cIevr56Kz09CgEROdWgVwQzswBwL3AlUAGsM7PVzrltYc2uBmZ7tzLgPqBssL5mVuTN06hllMXFGd+8/jzSEgPc/4d9NLV38c8fnEdyQsDv0kRkjIhkDWAxUO6c2+uc6wAeAZb2arMUeNCFvAYEzWxqBH2/D3wZ0NfTEWBm/O01c/n8kjk88UYlH77vFZ1ETkROiiQACoCDYc8rvGmRtOm3r5ldB1Q6594aYs0yBGbG7Utm88NlpVTWtfKB/3iJ1W8d8rssERkDIgkA62Na72/s/bXpc7qZpQJfBf5+0Bc3W25m681s/dGjRwctVvq25Jx8fnPbJZw1JYPbfvYmf/vLzbR1dvtdloj4KJIAqACKwp4XAr2/QvbXpr/pZwAzgbfMbL83/Q0zm9L7xZ1zq5xzpc650ry8vAjKlf4UBFN4ZPkFfPqyWfx07dtcf+/L7Dna5HdZIuKTSAJgHTDbzGaaWSJwA7C6V5vVwDJvb6ALgHrnXFV/fZ1zm51zk51zxc65YkJBsdA5dzhab0z6lhCI4ytXz+W/b3oX1Q1tfOA/XuJXb+r0ESKxaNAAcM51AbcCTwHbgZ8757aa2QozW+E1WwPsBcqB+4HPDtQ36u9Chuzysyez5vZLOHdaJnc8upG/eXwTrR3aJCQSS2w8HSRUWlrq1q9f73cZE0pXdw/fe3oX//n8Hs7Kz+Dej5/PmZMz/C5LRKLIzDY450p7T9eRwDEuPhDHl686m5/cspiapnY+8B8v8/iGCr/LEpFRoAAQAC6bk8ea2y9hfmEWX3zsLf7652/R0tHld1kiMoIUAHJSfmYyD3+qjNvecyZPvFnBdfe8rAvPi0xgCgA5RXwgji+89yweuqWMupZOlt77Eo+ue1snlBOZgBQA0qeLZ+ey5vaLWTg9m7/5xWZuf2QjNU3tfpclIlGkAJB+Tc5I5qFPlvGFK+ewZnMVl3/neR54aR+d3T1+lyYiUaAAkAEF4ozbrpjN7+64lAVFQf7xyW1ce/cfeKW8xu/SRGSYFAASkTMnp/PgLYtZ9ReLaO3s5mM/XMtnH95AxfEWv0sTkdOkAJCImRnvPXcKT3/+Mr5w5Rz+b8cRlnzvBX7wzG6dWE5kHFIAyJAlJwS47YrZPPvXf8IVZ+fz/Wd2seR7L/C7LYe1t5DIOKIAkNNWEEzh3o8v5KefKiM1McCK/9nAsgdep/yIjh0QGQ8UADJs7z4zlzW3XcLXP3AOGw/WcdW//4F/enIbjW2dfpcmIgNQAEhUxAfiuPmimTz3xT/hI4sK+dHL+7j8Oy/w2PqDuiC9yBilAJCoyk1P4lsfns+vPnsRhdkpfOnxTXzovlfYeLDO79JEpBcFgIyIkqIgT3zm3XznT0uoON7K9fe+zCceeJ1X99RqoFhkjND1AGTENbZ18uCrB/jvl/dR09TBgqIgKy47g/eek09cXF+XjRaRaOrvegAKABk1bZ3dPLahgvtf3Mvbx1o4Iy+NT196BtefX0BivFZGRUaKAkDGjK7uHtZsOczK5/ewraqBKZnJfPLimdxYNp30pHi/yxOZcBQAMuY453hxdw0rn9/Dq3tryUyOZ9mFxdx0UTG56Ul+lycyYSgAZEzbeLCOlc/v4alth0kMxPGnpYUsv+QMpuek+l2ayLinAJBxYc/RJla9sJcn3qygu8dx7fxprLhsFudOy/K7NJFxSwEg48rh+jYeeHkfD792gOaObi6ZncufvauIJXPzSU4I+F2eyLiiAJBxqb6lk/9Ze4CHXj3A4YY2MpPj+UDJND68qJDzi4KYaTdSkcEoAGRc6+5xvLKnhsc3VPDU1sO0dfYwKy+NDy8s5EMLC5ialeJ3iSJjlgJAJozGtk7WbK7i8Q0VrNt/HDO4+MxcPrKokPeeM4WURG0iEgmnAJAJ6UBtM794o5JfbKigsq6V9KR43j9/Kh9eVEjpjGxtIhJBASATXE+PY+2+Yzy+oYLfbqmipaObGTmpJzcRFWZrd1KJXQoAiRnN7V38bsthHt9Qwat7awEomzmJK8/JZ8ncfIpz03yuUGR0DSsAzOwq4AdAAPihc+5bveabN/8aoAW4yTn3xkB9zewbwFKgBzji9Tk0UB0KABmqiuMt/PKNSv530yF2VTcBcEZeGkvm5nPF3HwWTg8SH9B5iGRiO+0AMLMAsAu4EqgA1gE3Oue2hbW5BvgrQgFQBvzAOVc2UF8zy3TONXj9bwPOcc6tGKgWBYAMx9u1LTy7o5pntx9h7b5aOrsdwdQELj9rMlfMncylc/LITE7wu0yRqOsvACI589ZioNw5t9f7QY8Q+ua+LazNUuBBF0qT18wsaGZTgeL++p748PekAeNnW5SMS9NzUrn5opncfNFMGts6eXFXDc9ur+a5nUf45ZuVxMcZZbMmccXZoU1FOg2FTHSRBEABcDDseQWhb/mDtSkYrK+ZfRNYBtQDl/f14ma2HFgOMH369AjKFRlcRnIC186fyrXzp9Ld43jz7eM8s/0Iz26v5h+f3MY/PrmN2ZPTec/cySyZm8/C6dkEdO0CmWAiCYC+/up7f1vvr82AfZ1zXwW+amZfAW4Fvv6Oxs6tAlZBaBNQBPWKDEkgzigtnkRp8STuvPps3q5t4Znt1Ty7o5of/WEf//XCXoKpCbyreBJlMydxwawc5k7NVCDIuBdJAFQARWHPC4Heg7X9tUmMoC/AT4Hf0EcAiIy26Tmp3HLxTG65eCYNbZ38YVcNL+w6wtp9x3h6WzUAGUnxlBZnUzYrh8UzJzGvIIsEDSbLOBNJAKwDZpvZTKASuAH4WK82q4FbvW38ZUC9c67KzI7219fMZjvndnv9rwN2DPvdiERZZtimIgidpG7tvlrW7jvG2r21PLfzKAApCQEWzcimbOYkymblUFKURVK8jkiWsW3QAHDOdZnZrcBThHblfMA5t9XMVnjzVwJrCO0BVE5oN9CbB+rr/ehvmdlZhHYDPQAMuAeQyFgwJSuZpQsKWLqgAICapnZe98Jg7b5jfPfpXQAkxsdxflGQslk5lM2cxMLp2TpFhYw5OhBMJIrqWjp4fd+xUCjsO8bWQ/X0uNA4w5z8DOYVZDKvMMj8gizOmpKhU1vLqNCRwCI+aGjrZMOB46zff4zNlQ1srqjjeEsnAPFxxllTMphXkMW8wizmFwSZMyVdm44k6hQAImOAc47KulY2V9SzqbKeLZX1bKqop741FAoJAePsKZmcV5DF/MIs5hVkMSc/g8R4DTDL6VMAiIxRzjkqjreyqaKeTZV1J0Ohsa0LgMRAHGdNyWBOfgZnTUlndn7o8bSsZJ3tVCKiABAZR5xzvH2shU0VobWELYfq2VXdxNHG9pNt0pPiOXNyOnPy05mTn+EFQzpTMhUMcioFgMgEcLy5g91HmthV3Xjytru6idrmjpNtMpLjmT351FCYk5/B5IwkBUOMUgCITGC1Te3sqm5i95ETwdDE7urGkwPOAKmJAWbkpFGck0pxrnefk0ZxbprCYYIbzsngRGSMy0lP4sL0JC48I+fkNOccNU0d7K5uZPeRJvbXNrO/ppmdhxt5els1XT1//PKXkhBgRk4qM3PTmJGTxszcVO9e4TCRKQBEJigzIy8jibyMJN59Zu4p87q6ezhU18a+2mYO1Dazr6aZA7Ut7DzcyDPbq+nsfmc4FGanUJidSkEwhYLsFAqzUygIpjApLVEBMU4pAERiUHwgjuk5qd4pr/NOmdfV3UNVfZsXCs3sq2nhQG0zFcdbeW3vMZrau05pn5IQOCUQCsKCoig7hdz0JOJ04rwxSQEgIqeID8RRNCmVoknvDAfnHA2tXVTUtVB5vJWK461U1rVScbyFyrpW3jpYd8q4A4R2Y50WTGZKVjJTMpOZkpXClMyk0H1WMlOzkslNT9LZVX2gABCRiJkZWakJZKVmce60rD7bNLd3UVnX6gVECxXe4+qGNtYfOE51Q9Upm5ggdKqMvPSksJBIPhkO+ZmhaZMzk0hN1EdWNGlpikhUpSXFM8c7WK0vPT2OYy0dHK5vC90a/nhf3dBG+dEmXi6vobHXpiaAtMQAkzOTyUtPOjm+EX6b7N3npGmNIhIKABEZVXFxRm56ErnpSZxX0PdaBEBTe9cpIXG0sT10a2rnSEMb2w838OLu9pNHTJ/yGgaT0v4YCHkZSeSkJ5KbFrrPSU8iJy2R3PQkJqUlxuypNhQAIjImnTjS+czJ6QO2a+3opqapnSMnAqKx7WRQHG0MTd9V3UhtUwcd3T19/ozM5Hhy071w6BUSJ6ZNSkskOy2B7NTECXPxHwWAiIxrKYmBsEHr/jnnaGzv4lhTB7XN7dQ0dVDb1EFtUzu1zR3UNLVT29TB3pom1u3v4FhLB/0dJ5uRHB8KhNTEsPsEstMSmZSaGLoPm5+VkjAmN0kpAEQkJpgZmckJZCYnUJybNmj77h7H8ZY/hsSxlg6ON3dwrLmT4y0dHGvu4HhLB9UNbeyoaqC2uYP2rr7XMMxCV5fLTk0gmJpIMDW0JpGVEroPpiZ4t0SyT8xLTSAjKX5Ej7FQAIiI9CEQNlYBfQ9o99ba0X0yKE6GRHPovq61k+MtndR5obLnaBN1zZ19DnaH1xBMCYXDNz84jwtm5fTb9nQoAEREoiQlMUBBYuiAuEh1dvdQ39pJnRcOJ0KirqWTutbQ8/qWTrJSEqJerwJARMRHCYG4sDWN0TUxhrJFRGTIFAAiIjFKASAiEqMUACIiMUoBICISoxQAIiIxSgEgIhKjFAAiIjHKXH9nOxqDzOwocMDvOvqRC9T4XcQAVN/wqL7hUX3DN5waZzjn8npPHFcBMJaZ2XrnXKnfdfRH9Q2P6hse1Td8I1GjNgGJiMQoBYCISIxSAETPKr8LGITqGx7VNzyqb/iiXqPGAEREYpTWAEREYpQCQEQkRikAImBmRWb2nJltN7OtZna7N/0uM6s0s43e7ZqwPl8xs3Iz22lm7xuFGveb2WavjvXetElm9rSZ7fbus32s76yw5bTRzBrM7A4/l6GZPWBmR8xsS9i0IS8zM1vkLftyM7vbonQR137q+zcz22Fmm8zsl2YW9KYXm1lr2HJc6VN9Q/59jnJ9j4bVtt/MNnrT/Vh+/X2ujN7foHNOt0FuwFRgofc4A9gFnAPcBXyxj/bnAG8BScBMYA8QGOEa9wO5vaZ9G7jTe3wn8K9+1derrgBwGJjh5zIELgUWAluGs8yA14ELAQN+C1w9gvW9F4j3Hv9rWH3F4e16/ZzRrG/Iv8/RrK/X/O8Cf+/j8uvvc2XU/ga1BhAB51yVc+4N73EjsB0oGKDLUuAR51y7c24fUA4sHvlK+6zjJ97jnwDXh033s74rgD3OuYGO6h7xGp1zLwLH+njdiJeZmU0FMp1zr7rQf+KDYX2iXp9z7vfOuRNXEX8NKBzoZ4x2fQMYE8vvBO8b8keBnw30M0a4vv4+V0btb1ABMERmVgycD6z1Jt3qrY4/ELaqVgAcDOtWwcCBEQ0O+L2ZbTCz5d60fOdcFYT+2IDJPtYX7gZO/ccbK8sQhr7MCrzHo10nwC2Evu2dMNPM3jSzF8zsEm+aH/UN5ffp1/K7BKh2zu0Om+bb8uv1uTJqf4MKgCEws3TgF8AdzrkG4D7gDGABUEVolRJCq2G9jfT+thc55xYCVwOfM7NLB2jrR32hFzZLBK4DHvMmjaVlOJD+6vGlTjP7KtAFPOxNqgKmO+fOB74A/NTMMn2ob6i/T79+zzdy6pcQ35ZfH58r/Tbtp5bTrlEBECEzSyD0S3rYOfcEgHOu2jnX7ZzrAe7nj5soKoCisO6FwKGRrM85d8i7PwL80qul2ls9PLEqe8Sv+sJcDbzhnKv26h0zy9Az1GVWwambYUa8TjP7BPB+4OPeKj/eZoFa7/EGQtuH54x2fafx+/Rj+cUDHwIeDavbl+XX1+cKo/g3qACIgLe98EfAdufc98KmTw1r9kHgxN4Gq4EbzCzJzGYCswkN0oxUfWlmlnHiMaGBwi1eHZ/wmn0C+LUf9fVyyjevsbIMwwxpmXmr6I1mdoH3d7IsrE/UmdlVwN8A1znnWsKm55lZwHs8y6tvrw/1Den3Odr1eZYAO5xzJzeb+LH8+vtcYTT/BqMxmj3Rb8DFhFapNgEbvds1wEPAZm/6amBqWJ+vEvoWsZMo7TUwQH2zCO0d8BawFfiqNz0HeBbY7d1P8qO+sNdMBWqBrLBpvi1DQkFUBXQS+hb1ydNZZkApoQ+6PcA9eEfYj1B95YS2A5/4O1zptf2w97t/C3gD+IBP9Q359zma9XnTfwys6NXWj+XX3+fKqP0N6lQQIiIxSpuARERilAJARCRGKQBERGKUAkBEJEYpAEREYpQCQEQkRikARERi1P8D2amdYgl52nEAAAAASUVORK5CYII=\n",
       "text/plain": [
        "STEP: 2000 MSE: 0.0023767943494021893\n"
       ]
      },
      "metadata": {
       "needs_background": "light"
      },
      "output_type": "display_data"
     }
   ],
   "source": [
    "STEPS = 2000\n",
    "LEARNING_RATE = .02\n",
    "\n",
    "\n",
    "Xf = make_features(X)\n",
    "n_weights = Xf.shape[1]\n",
    "\n",
    "W = tf.Variable(np.zeros((n_weights, 1)), dtype=tf.float32)\n",
    "\n",
    "# For plotting\n",
    "steps, losses = [], []\n",
    "plt.figure()\n",
    "\n",
    "\n",
    "for step in range(1, STEPS + 1):\n",
    "\n",
    "    dW = compute_gradients(X, Y, W)\n",
    "    W.assign_sub(dW * LEARNING_RATE)\n",
    "\n",
    "    if step % 100 == 0:\n",
    "        loss = loss_mse(Xf, Y, W)\n",
    "        steps.append(step)\n",
    "        losses.append(loss)\n",
    "        plt.clf()\n",
    "        plt.plot(steps, losses)\n",
    "\n",
    "\n",
    "print(\"STEP: {} MSE: {}\".format(STEPS, loss_mse(Xf, Y, W)))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
     {
      "data": {
       "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA+BUlEQVR4nO3dd3hUxfrA8e9seoeEUAOE3nsIXcBKkaaggHSUJtZ7vRfLvfZ21Z+ioohSBSnSBKQJ0ntCL6EHktASQklIT+b3x4lICckm2c1ml/fzPHlIduecfVkOb2bnzLyjtNYIIYSwfyZbByCEEMIyJKELIYSDkIQuhBAOQhK6EEI4CEnoQgjhIJxt9cKlSpXSwcHBtnp5IYSwS+Hh4XFa68CcnrNZQg8ODiYsLMxWLy+EEHZJKXXmXs/JkIsQQjgISehCCOEgJKELIYSDsNkYuhDCMaWnpxMdHU1KSoqtQ7Fr7u7uBAUF4eLiYvYxktCFEBYVHR2Nj48PwcHBKKVsHY5d0lpz+fJloqOjqVKlitnHyZCLEMKiUlJSCAgIkGReCEopAgIC8v0pRxK6EMLiJJkXXkHeQxlyERZzIzWDyMs3iIpP4kpSOgkp6SSlZeJsUrg4mfB0c6aMjxulfd0JDvCkhKerrUMWwqFIQhcFduJSIltOxBF+5grhZ64QczU5X8eX8XWjdllfQiqXpFW1ABoGlcDVWT40iqKzfv16XF1dad26dYHP4e3tTWJiogWjKjhJ6CJfTlxKYMHuGFYdusCp2BuAkZhDKvvTv0UlggO8qBzgib+XK74eLni6OJGRpcnIyiIxJYNLCalcuJbCqbhEIi4kcPjcdb744xj8AZ6uTnSoFUiXBuV4sHZpPF3l8hTWtX79ery9vQuV0IsT+R8j8pSRmcXvB84za/tZdkbG42RStKzqz5DWwXSsVZqgkh65jve5mhSumPB0daa0rzv1K/gBZW4+f+VGGjtOX2bT8ThWHbrI8gMX8HBxokuDcgxoWYnGFUvImKzIl549exIVFUVKSgovvfQSI0aMYOXKlbzxxhtkZmZSqlQpJk+ezMSJE3FycmLmzJl88803TJ48mccff5zevXsDf/e+ExMT6dGjB1euXCE9PZ0PPviAHj162PhveTdJ6OKe0jKyWLQnmgnrTnI2PonKAZ6M61yb3s2CKOXtZrHXKenlSqf65ehUvxzv9ajPztPxLNl3jiV7Y1iwO5p65X15tl0VujUsj7OTDMnYk3eXHuLwuesWPWfd8r683a1erm2mTJmCv78/ycnJNG/enB49evDcc8+xceNGqlSpQnx8PP7+/owaNQpvb2/++c9/AjB58uQcz+fu7s6iRYvw9fUlLi6Oli1b0r1792LX0ZCELu6itWb14Yt8+PsRzsYn0TDIj/88HsJDtUtjMln3AnYyKVpVC6BVtQDe7FqHRXti+HlbJK/M3cdXa44zpkM1ejUJkrF2kauvv/6aRYsWARAVFcWkSZN44IEHbs7p9vf3z9f5tNa88cYbbNy4EZPJRExMDBcvXqRs2bIWj70wJKGL25y4lMg7Sw6x+UQcNUp7M2VICB1rlbZJT8TbzZmBLSvzTGgl/jhykW//PMG/Fxzg23UnGNepDl0alC12PSRxu7x60tawfv161qxZw7Zt2/D09KRDhw40atSIo0eP5nmss7MzWVlZgJHE09LSAJg1axaxsbGEh4fj4uJCcHBwsVwJK90cAUBmluaHDSfp8vUm9kdf5Z1udVnxUjserF3G5knTZFI8Vq8sS8a2YeqQ5ni5OvP8L7vpPXEbu89esWlsovi5du0aJUuWxNPTk4iICLZv305qaiobNmzg9OnTAMTHxwPg4+NDQkLCzWODg4MJDw8H4LfffiM9Pf3mOUuXLo2Liwvr1q3jzJl7VrC1KaW1tskLh4SEaKmHXjycvZzEy3P3sPvsVR6tW4YPezUg0MdyY+SWlpml+TUsis9XHyMuMZV+oRUZ17kOfh7m17wQ1nPkyBHq1Kljs9dPTU2lZ8+exMTEUKtWLWJjY3nnnXdITk7mjTfeICsri9KlS/PHH39w7Ngxevfujclk4ptvvqFmzZr06NGDrKwsHnroIb755hsSExOJi4ujW7dupKen07hxY7Zs2cKKFSsIDg626rTFnN5LpVS41jokp/aS0O9zfxy+yKvz9qKA93vWp3uj8jbvkZsrMTWD8WuOMXnzaQK83Xi3ez0615dhGFuzdUJ3JPlN6GYNuSilOimljiqlTiilxuXSrrlSKlMp1TtfUYsil5GZxccrjvDcjDAqB3jy+4vt6NG4gl0lQ283Z97sWpffnm9LaR83xszazaiZ4VxOTLV1aELYRJ4JXSnlBEwAOgN1gX5Kqbr3aPcpsMrSQQrLSkzNYPj0MH7YcIpnWlRi/qjWVPT3tHVYBdYgyI/fnm/DuM61WRcRS6fxm9h4LNbWYQlR5MzpoYcCJ7TWp7TWacAcIKcZ9S8AC4BLFoxPWNiFayk8NXEbm0/E8VGvBnzYqwHuLk62DqvQnJ1MjGpfjcXPt6GkpwuDpuzk3aWHSEnPtHVoQhQZcxJ6BSDqlp+jsx+7SSlVAegFTMztREqpEUqpMKVUWGys9KCK2rGLCfT6bgtnLt9g8uAQ+reoZOuQLK5ueV+WjG3LkNbBTN0SSe+JW4mKT7J1WEIUCXMSek6DqnfeSf0K+LfWOtfukNZ6ktY6RGsdEhgYaGaIwhIOnbvG0z9sIzNLM29UKzrUKm3rkKzG3cWJd7rX46dBIZy5nES3bzez/qh8cBSOz5yEHg1UvOXnIODcHW1CgDlKqUigN/CdUqqnJQIUhbcv6ir9f9yBh4sT80a2ol55P1uHVCQerluGpWPbUtbXnaHTdvHVmmNkZdlmVpcQRcGchL4LqKGUqqKUcgX6AktubaC1rqK1DtZaBwPzgTFa68WWDlbk3+6zVxjw0w58PZyZO7IVwaW8bB1SkQou5cWiMW3o1aQCX605zsiZ4SSlZdg6LGFH1q9fz+OPPw7AkiVL+OSTT+7Z9urVq3z33Xf5fo133nmHzz//vMAx/iXPhK61zgDGYsxeOQLM01ofUkqNUkqNKnQEwmqOnL/OkCk78fd2Ze6IVnY9k6UwPFyd+KJPI97pVpe1Ry7SZ+I2Llwrfsu2RdHKzMz/DfPu3bszbtw9Z24XOKFbilnz0LXWy7XWNbXW1bTWH2Y/NlFrfddNUK31EK31fEsHKvInMu4GAyfvxNPVmZnDW1C+hIetQ7IppRRD2lRh8uDmnLmcRI8JmzkYc83WYQkriYyMpHbt2gwePJiGDRvSu3dvkpKSCA4O5r333qNt27b8+uuvrF69mlatWtG0aVP69Olzc8XnypUrqV27Nm3btmXhwoU3zztt2jTGjh0LwMWLF+nVqxeNGjWiUaNGbN26lXHjxnHy5EkaN27Ma6+9BsBnn31G8+bNadiwIW+//fbNc3344YfUqlWLhx9+2Kw6M+aQ4lwO6MK1FAZM3kFmVhZz7uOeeU461i7N/NGtGD4tjD4TtzG+b2MerVe8KuY5lBXj4MIBy56zbAPofO9hj78cPXqUyZMn06ZNG4YNG3az5+zu7s7mzZuJi4vjiSeeYM2aNXh5efHpp5/yf//3f/zrX//iueee488//6R69eo8/fTTOZ7/xRdfpH379ixatIjMzEwSExP55JNPOHjwIHv37gVg9erVHD9+nJ07d6K1pnv37mzcuBEvLy/mzJnDnj17yMjIoGnTpjRr1qzQb40U53IwN1IzGDZtF1dupDF9WCjVS/vYOqRip3ZZXxY/34ZaZX0YNTOcOTvP2jokYQUVK1akTZs2AAwYMIDNmzcD3EzQ27dv5/Dhw7Rp04bGjRszffp0zpw5Q0REBFWqVKFGjRoopRgwYECO5//zzz8ZPXo0AE5OTvj53T3ZYPXq1axevZomTZrQtGlTIiIiOH78OJs2baJXr154enri6+tL9+7dLfJ3lh66A8nM0rw0Zw8RF64zZUhzGgaVsHVIxVagjxu/PNeCMbN2M27hAS7fSGNMh2p2VfrALpjRk7aWO/8t//rZy8uYGKC15pFHHmH27Nm3tdu7d6/FrgOtNa+//jojR4687fGvvvrKKtea9NAdyMfLj7DmyCXe6V7PoeeZW4qnqzM/DgqhV5MKfLbqKO8uPSzTGh3I2bNn2bZtGwCzZ8+mbdu2tz3fsmVLtmzZwokTJwBISkri2LFj1K5dm9OnT3Py5Mmbx+bkoYce4vvvvweMG6zXr1+/qxzvY489xpQpU26OzcfExHDp0iUeeOABFi1aRHJyMgkJCSxdutQif2dJ6A5i1o4z/LT5NENaBzOoVbCtw7EbLk4mvujTiOFtqzBtayQvz91LWkaWrcMSFlCnTh2mT59Ow4YNiY+Pvzk88pfAwECmTZtGv379aNiwIS1btiQiIgJ3d3cmTZpE165dadu2LZUrV87x/OPHj2fdunU0aNCAZs2acejQIQICAmjTpg3169fntdde49FHH6V///60atWKBg0a0Lt3bxISEmjatClPP/00jRs35sknn6Rdu3YW+TtL+VwHEH4mnqd/2E7bGqWYPLg5TlbeJs5iMjMgOR6SLkN6MmSkQkb2n1mZ4OQCJicwuRjfu/mCRwlwLwGuXmDBj6xaayZuOMWnKyN4uE4ZJjzTBDdn+69xYwvFoXxuZGQkjz/+OAcPHrRpHIWV3/K5MoZu52ITUhkzazcVSnowvm+T4pXM05Mh7hjEn4IrkRB/Gq6choQLcCMOkq9wdxUJM5lcwDMA/IKMrxIVwa8iBFSDwNrgWyFfCV8pxegO1fB2c+I/vx1i5M/hTBzQzCEKl4n7hyR0O5aRmcULs3dzLTmdqUNCbbtjT9oNiA6D83uNaWoXDhjJXN8yfOFZCvyrGAk3OBC8AsGrFHiUNHrczu7ZX25Gzzwrw+jFZ6VDZhqkJhi/BJKvGn/eiIVr0cZrHV0BmbfUQXfzhcBaULouVGgKQc2N1zXlnqAHtgrGxcnE64sO8Oz0MH4cFIKHqyR1exMcHGz3vfOCkIRuxz5bfZTtp+L5ok8j6pb3LdoXT02AM1vhzBbjz3N7jAQM4BtkzBWu0x3K1DN6zSUqg7sVY9QaEi/B5eNw6QjERsClCDj8G+yebrRx9YbyTaBSK6jawUjyzq53napvaCVcnEy8Nn8fQ6buZMqQ5ni5yX+V/NBay4yhQirIcLhcpXbqz4iLNzeoeLJZUNG86OWTcHw1HFsJkVuMnrPJxegBt34BKrWGCs3AK6Bo4rmVUuBTxvgKvmU2g9bGkE90GETvgpgw2PQ5bPwfuHhC5dZGcq/VxfjFk+3JZkG4OJt4Ze5eBk3ZyfRhoXhLUjeLu7s7ly9fJiAgQJJ6AWmtuXz5Mu7u7vk6Tm6K2qFLCSl0/moTpX3dWfx8a+vevIs7Dgfmw6GFxhAKQKlaUPNRqP4IVAwFFzsrK5B81fhkcWq98fXX3yuwNtTuanyVawImEysOnOeF2XtoWqkk04Y1x9NVknpe0tPTiY6OJiVF6uUUhru7O0FBQbi43D6UKptEO5CsLM3QabvYfuoyy15oS40yVlgJmnAR9s+Fg/Ph/D5AGb3eOt2gxqPGOLgjuXrWGIOPWGZ88tCZxg3WBn2gUV+Wnffhxdl7aFk1gClDmsuNUmFTktAdyJTNp3lv2WHe71mfgS1znh9bIFlZcHo9hE2Fo8uN8fDyTY2kVq8X+Jaz3GsVZ0nxcGyV8cvs5J/GTd1yjdkf0Imh4cHUr1GNSYOayZRGYTOS0B3EkfPX6fHtFh6oWYofB4VYZnwy5bpx03DXT8bUQs8AaNwfmg6GUjUKf357lnDRSOz75sCF/WSaXFiSHkpE0NP8Y9gAXKWnLmxAEroDSM3IpPs3W4hPSmPlS+0I8HYr3AmvxcCO7yF8OqReh8ptIGSYMaziXMhzO6JLRyB8GmnhM3HNSCTatSrlHh6LU+O+xpRLIYqIJHQH8Pmqo3y77gRThzSnY+1C1Gm5fBI2fg4H5hkzQOr1hFZjjZkqIm+piWxaNBH/wzOoZzqD9iiJCh0BoSOMOfVCWJkkdDt3MOYaPSZsoWfjCnzxVKOCneRKJGz8DPbOBidXaDYEWo6GkhYch7+PfLv2GOvWLOPD0n9S+9omcPaAJgOg9VgoGWzr8IQDk6X/diwtI4vX5u/H38uV/z5eN/8nSLgI6z+GPT+DcjJ6km1fMeZriwJ7/sEaXEnuRKfNtXiv9UsM0oshfBqETYaGfaH9vxxvNpAo9iShF3Pfrz/JkfPXmTSwGX6e+Vjan54M276FTV8ay+abDYF2/wDf8laL9X6ilOLNLnW4mpTOf7dGQ4/XGPTym7D1WyOpH5hn9NgfeM2oNSNEEZCEXoxFXLjOt+uO071RefO3SdMaDi6ANe/AtSio/Tg88t5tqyCFZZhMik+fbMC15HTeXnIIP4/G9Oj0kbFqdtMXRo997y/QbKiR2L0DbR2ycHBSD72YysrSvL7wAL7uLrzTvZ55B8UehWldYcFwo+DV4GXQd5YkcytydjLxbf8mNA/25x/z9rHu6CVjzn7Xz+HF3dCorzEl9JumsPkrozSwEFYiCb2YmrMrij1nr/Jm1zr4e91dQOo26cnw5wfwfRu4eAi6jYcR66GKZYrmi9y5uzjx0+AQapX1YfTMcMIi440nSlSC7t/AmO1GzZg1b8O3zeHQYuOTlBAWJgm9GIpLTOWTFUdoWdWfXk0q5N741Ab4vrUxg6X+EzA2zBgvz6NMrLAsX3cXpg8LpZyfB8Om7eLYxb+3ISOwJvSfCwMXg5sP/DoYpnY2yv4KYUGS0Iuhj5YfITk9kw961r/3atC0JFj+GszI3i184GJ4YpKM09pQKW83ZgwLxc3FicFTdnLh2h3Fqap1hJEbodvXRtGzH9rDyjeMUsRCWIAk9GJm+6nLLNwdw4gHqlK99D0Kb0XthIltYeckaDEaRm0xkoWwuYr+nkwb2pzryekMmbqT6ynptzcwOUGzwTB2FzQdBNsnwLehRt12GYYRhSQJvRhJy8jircUHCSrpwdiOOdRRyUyHte/BlMeMqYiDl0LnT8DVs+iDFfdUr7wfEwc248SlREbOCCc1I/PuRp7+0O0rGL7GqJ8zbxD88pSxA5MQBSQJvRiZsS2SE5cSebd7vbu3PbsaBVO7GNPhGvWH0VuhygO2CVTkqV2NQP7XuyHbTl3mtV/3k5V1j953xebGDezHPoLIzfBdK9g9Q3rrokAkoRcTlxNTGb/2OO1rBvJQnTtWcUYsN4ZYLh2BJydDzwnW3c5NWMQTTYN47bFaLNl3jk9XRty7oZMztHre+CVdtiEseQFmPim9dZFvktCLiS/+OEZSWib/ebzO3w9mpBk3zeb0M6bAjdwADXrbLkiRb2M6VGNgy8r8sPEU07aczr2xfxVjGK3L53B2O0xoaVTDlN66MJMk9GLgyPnrzNl5lkGtKv99IzQxFn7uadw0a/4cDP9DFgjZIaUU73Svx6N1y/DussOsOHA+9wNMJgh9DkZvgfKNYemLMKc/3LhcJPEK+2ZWQldKdVJKHVVKnVBKjcvh+R5Kqf1Kqb1KqTClVNucziPuprXmvaWH8fNw4eWHahoPntsLkzpATDg88aOx6tAlf5vFiuLDyaT4ul8TmlQswUtz97LzdHzeB/lXgUFLjLH1E2uMtQan1ls9VmHf8kzoSiknYALQGagL9FNK3Vn2by3QSGvdGBgG/GThOB3WqkMX2XbqMq8+UtMovnVgvjGLBQ3DVkLDp2wdorAAdxcnJg9uTlAJD56bEcbJ2MS8DzKZjLH1Z9ca90xm9IQ//msMxQmRA3N66KHACa31Ka11GjAH6HFrA611ov67sLoXIIN+ZkjNyOTD5YepVcaHfs0rwtr3jTos5ZsYMx/KN7F1iMKCSnq5Mn1YKC5OiqFTd3E50cy6LuUawogNxgrgLeNh8iPGRiVC3MGchF4BiLrl5+jsx26jlOqllIoAfsfopd9FKTUie0gmLDY2tiDxOpSft50hKj6ZtzpVw3nJaNj0OTQZaHzU9i7ErkSi2Kro78mPg0K4eD2FZ2eEkZKewxz1nLh6GvPWn55pbFYyqQMcWWbFSIU9Mieh57T2/K4euNZ6kda6NtATeD+nE2mtJ2mtQ7TWIYGB9/cS9esp6UxYd4LHqrnTbsdI2D8XHnzLKObknEcxLmHXmlQqyfi+jdkbdZVX5+299xz1nNTpZpQP8K8Kc58xhmAyM6wXrLAr5iT0aKDiLT8HAefu1VhrvRGoppSSDRZzMWnDKdyTLjA+aRyc3Qa9fjBqZt+rdotwKJ3ql+ONznVYfuBC7nPUc1KyMgxbZdRZ3zIeZvQwdqYS9z1zEvouoIZSqopSyhXoCyy5tYFSqrrKriKllGoKuAIyz+oeLl1PYc3mzSz3fg/3pAswYIFRN1vcV55tV+XmHPVZO87k72AXd2MIpudEYzbUD+3gzDarxCnsR54JXWudAYwFVgFHgHla60NKqVFKqVHZzZ4EDiql9mLMiHla22r3aTswZ9kKZprewddFw9DlULWDrUMSNqCU4u1udelYK5D//naI9Ucv5f8kjfvBc2vB1QumdzMWIon7lrJV3g0JCdFhYWE2eW1bijmwEe/5fcHVC7+Ry6FUDkW4xH3lRmoGfSZu48zlG/w6qjV1yxegrEPyFfh1KJxaBy1GwaMfGiUFhMNRSoVrrUNyek5Wihal05sIWPgU1/Amc4gkc2HwcnNmypDm+Hq4MGzaLs5fS87/STxKwjPzoeUY2DERZvU2kry4r0hCLyqnNpA180nOZAawssU0/CtIMhd/K+vnzpQhzUlMzWDYtDASUwswc8XJGTp9DN2/NSo3/vggxB6zfLCi2JKEXhQit8DsvsSYyjHa+V36PxRq64hEMVSnnC8TnmnKsYsJjP1lNxmZWQU7UdOBRpGvlOsw+WEjuYv7giR0azu7A2b1IdmzHL0S/k3fDk3wdpOxTZGz9jUDeb9HfdYfjeXtJYco8D2uyq3guT/Buwz83MsoKSEcniR0a4oJN8YyfcryT8/3wTuQAS0r2zoqUcz1b1GJUe2rMWvHWX7cdKrgJ/prvnqFEKOkxOavpBSvg5OEbi0XDho9I09/9nScwe+nYVT7ani6Su9c5O1fj9Wia8NyfLQ8Iu+Su7nx9IeBi6DeE7Dmbfj9H7Ky1IFJdrGGK5Ew8wlw9YZBS/j013ME+rhJ71yYzWRSfNGnEeevJvPy3L2U8XOnaaWSBTuZi7ux01WJisbK0uvnoPcU2YvWAUkP3dJuxMHPT0BGKgxYwNZ4L7afimdMh2q4uzjlfbwQ2dxdnPhxUAhl/dx5bnoYZy8nFfxkJhM88p6xG9Kxlcanx+SrFotVFA+S0C0pNRFm9YHrMdB/HjqwNl/+cYwyvm70C61k6+iEHQrwdmPqkOZkas2QaTu5mlTIWuihz0Gfqcb9nemPQ2IBVqeKYksSuqVkpsO8QXB+H/SZBpVasOXEZXZFXuH5jtWldy4KrGqgNz8OCiH6SjIjZoSbX3L3Xur1gv5zIO4ETOkEV6PyPkbYBUnolqA1/P4qnFxrFEyq1RmtNV+tOUY5P3eebl4xz1MIkZvmwf580acROyPjeW3+/vyV3M1J9Ydh0GJjiHDKY7IAyUFIQreEbRNg9wxo909oOgiAHafjCTtzhVHtq+HmLL1zUXjdGpVnXOfaLN13js9WHy38CSu1hKG/Q2YaTO1k7GUr7Jok9MI6ugJWvwV1e0DHN28+PGHdCUp5u0rvXFjUyAeq8kyLSny//iS/7Dhb+BOWbWDMVXfxgundITq88OcUNiMJvTAuHID5w6FcI6Mutcl4O/dFXWXT8TiGt60qY+fCopRSvNu9Hg/WLs1/fjvIuoKU3L1TQDWjjLNnSfi5J0TtKvw5hU1IQi+oxEvwS19w94N+c26b0/vtuhP4ujszoKXMbBGW5+xk4pt+TahTzofnZ+3mYMy1wp+0REUY8jt4BhhTGs/uKPw5RZGThF4QmRkwfxgkxUG/2eBb7uZTEReu88fhiwxpUwUfdxcbBikcmZebM1MGN6ekpytDp+0i5moBSu7eyS/I6Kl7lzYWxskOSHZHEnpBrH0XIjfB419B+ca3PfX9+pN4ujoxtHWwLSIT95HSvu5MHdqclPRMhk7dybXk9MKf1Le80VP3KQcznzQqhQq7IQk9vw7/Blu/hpDhxvZft4iMu8HSfecY0LIyJb1cbRSguJ/ULOPDDwOacTruBqNnhpOWUcCSu7fyLQdDloFfBaO4nPTU7YYk9PyIPQaLx0BQc+j0yV1PT9xwEmcnE8+2rWKD4MT9qnX1UnzyREO2nrzMuIX7C15y91Y+ZY2eul+Qsfo5Rma/2ANJ6OZKTYS5A8DZHfpMB+fbe+DnryWzYHc0T4dUpLSvu42CFPerJ5sF8crDNVm4O4av1hy3zEm9S8Og38ArwKhPdOGgZc4rrEYSurlW/AvijhlV6vwq3PX0lM2nydIw4oGqNghOCHjxoer0bhbE+LXH+TXMQsv5fcvDoCXg6gUzesiK0mJOEro5DsyHvbPggdegavu7nr6eks7snVF0aVCOiv5SklTYhlKKj59oQNvqpXh94QE2H4+zzIlLVjaSujLBjO4QX4hNN4RVSULPS/xpWPYKVGwB7f+dY5PZO86SmJrBSOmdCxtzcTLx3YCmVC/tzeiZ4Rw5f90yJy5V3Rh+yUiF6T3gWrRlzissShJ6bjLTYcGzgIInfzJ2Vb9DWkYWU7dE0rpaAPUr+BV9jELcwdfdhSlDmuPl5szgKTuJii9EHfVblakLAxdCylVj+OXGZcucV1iMJPTcrP8YYsKg+3gokfOqz6X7znHhegrPSe9cFCPlS3gwY3goKemZDJ6yk/gbhayjfvPETaD/PKOHPqu3MVlAFBuS0O8laids/hKaDDDqR+dAa82Pm05Rq4wPHWoGFnGAQuSuZhkfJg9pTszVZIZO20VSmoX2Eq3cCnpPNWr/zxsIGRb6ZSEKTRJ6TtKTYfFo8K0Aj318z2Ybj8cRcSGBZ9tVQSlVhAEKYZ7mwf58068JB6KvMmbWbtIzLbDwCKB2F+g2Hk7+afxfybLQeUWhSELPyZ8fwOUT0P0bcPe9Z7MfN56ijK8bPRrfPY1RiOLi0Xpl+bBXA9YfjeXfCyy08Aig6UB46G04OB9WvWFs9CJs6u67fPe7M9uMDStChkO1jvdsdjDmGptPxPHvTrVxdZbfi6J46xdaiUvXU/lyzTECfdx4vXMdy5y47StG5dEd3xsLkdq9apnzigKRhH6rtBvw2xijlOgj7+Xa9KdNp/BydaJ/CymRK+zDiw9V51JCCj9sOEVpH3eGW6JEhVLw2EdG5dG174J3GWjyTOHPKwpEEvqt1n1kLJoYvAzcvO/ZLOZqMkv3n2dI62D8PKRErrAPSine61Gfy4lpvL/sMKW8XS0zXGgyQY/vjP1Jl75orKSu2qHw5xX5ZtZYgVKqk1LqqFLqhFJqXA7PP6OU2p/9tVUp1cjyoVrZ+X2w/TtoNhSqtMu16fStkQAMkyJcws44mRRf9W1MaBV//vnrPjYdj7XMiZ1d4anpUKomzB0ElyIsc16RL3kmdKWUEzAB6AzUBfoppere0ew00F5r3RB4H5hk6UCtKivTWA3qGQAPv51r0xupGczeeZZO9ctSoYRHEQUohOW4uzjx46AQqgV6M/LncMLPXLHQif2MOeou7kaFxoSLljmvMJs5PfRQ4ITW+pTWOg2YA/S4tYHWeqvW+q+rYjsQZNkwrSxsilEe9LGPwKNkrk0X7o4mISWDYW2CiyY2IazAz8OFGcNCCfRxY+jUnRw+Z6ESASUqQv+5xpj67L6QZqFVqsIs5iT0CsCtpduisx+7l+HAipyeUEqNUEqFKaXCYmMt9FGvsBIuwNr3jDG/Bn1ybZqVpZm6NZKGQX40rZR74heiuCvt687M4S3wcnNm0JQdnIq10KrP8k3gyclwbg8sfM74BCyKhDkJPacVMzlOOFVKdcRI6DlWsdJaT9Jah2itQwIDi8nKylVvGAWHuv6fccc+FxuPx3Iq9gZD2wTLQiLhECr6e/Lz8BZoDQN+2mGZvUnBWHjU6ROIWAar/2OZc4o8mZPQo4GKt/wcBJy7s5FSqiHwE9BDa20fVXvObIWDC6DtyxBQLc/mU7dEEujjRtcG5a0fmxBFpHppb6YPCyUhNYOBP+0gNiHVMiduOQpajILtE2DXZMucU+TKnIS+C6ihlKqilHIF+gJLbm2glKoELAQGaq3towJ+VhasHGcs72/zcp7NT1xKZMOxWAa0qCwLiYTDqV/Bj6lDmnP+WgqDpuzkWpIFNpwG475UjUeNDWJOb7LMOcU95ZmZtNYZwFhgFXAEmKe1PqSUGqWUGpXd7L9AAPCdUmqvUirMahFbyr5fjKmKD78LrnlvSjF9aySuTiZZSCQcVkiwPz8MbMaJSwkMnbaTG6kWKOZlcjJKT/tXhXmDjP0FhNUoi9V1yKeQkBAdFmajvJ+aAN80gxKVYfjqPMfOryWn0+rjtXSuX44vnrK/KfZC5MeKA+d5/pfdtK5Wip8Gh+Du4lT4k14+CT8+aGxpN3w1uPkU/pz3KaVUuNY6JKfn7s+xg01fQOJF46aNGTc35+2KIiktk6EyVVHcBzo3KMf/ejdiy8k4Rv4cTmqGBWapBFSDPtMg9igsHCnVGa3k/kvo12Jg23fQ8GkIapZn88wszfRtkYRW8ZcdicR9o3ezID7u1YANx2IZPXO3ZZJ6tY7Q6WM4+jus+6Dw5xN3uf8S+oZPAA0d3zSr+R+HLxJ9JVkWEon7Tt/QSnzYqz5/Rlxi7C97SMuwQK86dAQ0HWR8Sj4wv/DnE7e5vxJ63AnYMwtChhk7mZth2tbTVCjhwSN1y1o5OCGKn2daVOa9HvX44/BFXpy9p/AbZCgFXb6ASq3gt+eNiQnCYu6vhL7uA3B2h3b/NKv50QsJbD8Vz6BWlXEyyUIicX8a1CqYt7vVZeWhC7w8Zy8ZhU3qzq7w1M/g4Q9zB0JSvGUCFfdRQj+3Fw4tglZjwNu8Vaozt5/B1dnEUyEV824shAMb2qYKb3Wtw+8HzvPKvH2FT+regfDUDLh+Lrs8gNwktYT7J6Gvyy681foFs5onpKSzcHc03RqWp6SXq5WDE6L4e7ZdVV7vXJul+87xqiWSesXm0PlTOLEm+96WKKz7Y4OL8/vg+Cro+JZR4tMMi/fEcCMtk4GtzBtrF+J+MLJ9NbI0fLoygrSMLL7u16RwK6dDhhmVTjd8CuWbQq1Olgv2PnR/9NA3fQFuvhD6nFnNtdb8vP0MDYP8aFyxhHVjE8LOjO5Qjf8+boypj5oZTkp6IaY0KgVdv4ByjWDhCGMBkigwx0/osUfh8BIjmXuUMOuQHafjOXYxkQEtpXcuRE6Gta1yc0rjs9PDSEorRJkAFw/jJqnJZNwklRrqBeb4CX3zl8YF03KM2Yf8vP0Mfh4udG8kVRWFuJdnWlTm8z6N2HoyjiFTdpGQUoiCXiUrGzVfLh2GpS+BjUqS2DvHTuhXImH/PGOfUK9SZh1y6XoKqw5e4KmQIMvUsBDCgfVuFsT4vk0IP3uFgZMLWaWx+sPGgr8D84xdxES+OXZC3/49KBO0Hmv2IbN3RpGRpXmmhQy3CGGObo3K8/0zTTl87jr9ftxOXGIh6qm3+4eR2Fe+Duf3Wy7I+4TjJvSU68aq0Hq9jApvZkjPzOKXnWdoXzOQ4FJeVg5QCMfxaL2yTBrUjFNxifT+fitR8QUcBzeZoNcP4OkPvw4x/h8LszluQt87C9ISjF1TzLTm8EUuXk9loNwMFSLfOtQqzaxnW3IlKZ0nvt/KkfMFTMZepaD3FLhyWsbT88kxE3pWJuz4AYJCoULeFRX/8vP2M1Qo4UHH2qWtGJwQjqtZ5ZL8OqoVTkrx1A/b2Hm6gMv6K7eGB9+CQwshfKplg3RgjpnQj682frvno3d+4lICW09e5pmWlaRuixCFULOMDwvGtCbQx42Bk3fwx+GLBTtRm1eg2kOwYhxcOGDZIB2UYyb0HRPBpzzU6W72ITO3n8XVycTTUrdFiEKrUMKD+aNaU7usD6NmhjMvLCr/J7l1PH3eYGOnMZErx0vo8afh1HpoNgScXMw6JCktgwXh0XRpUJYAbzerhifE/cLfy5VfnmtJ62oB/Gv+fsavOU6+t7z0DoQnJ2ePp78s4+l5cLyEvncWoKBxf7MPWbrvHAmpGbIyVAgL83JzZvLg5jzRtAJfrjnGP37dl/+NMoLbQMc34OB82POzdQJ1EI6V0LMyYe8vUO1BKGH+0MkvO6OoWcabZpVLWjE4Ie5Prs4mvujTiFcersnC3TEMmrIj/wuQ2v4DqjwAK/5tbFQjcuRYCf3kn3A9BpoONPuQQ+eusS/qKv1CK6HM2DBaCJF/SileergGXz7diN1nrtLr+y2cvZyPuep/jac7u8GC4ZCRZr1g7ZhjJfTdM4xdUGp1MfuQ2TvP4uZs4okmQVYMTAgB0KtJED8PDyX+Rhq9vtvC7rNXzD/Ytzx0/wbO74V1H1otRnvmOAk9+QocXQENnzZ+i5vhRmoGi/eco2vDcvh5mncDVQhROC2qBrBwdGu83Z3pO2k7C8KjzT+4TjdjwsOW8XB6o9VitFeOk9AjlkNWOjTobfYhy/afIzE1g2daVLJiYEKIO1UN9GbRmDY0q1SSf/y6j/eWHjZ/B6THPoKA6rBwpOxHegfHSeiHFkKJSvlaGfrLjrPULONN00pyM1SIoubv5cqM4aEMaR3MlC2nGTx1J1dumDE27uoFvSfDjVhY8oJMZbyFYyT0pHhj7nm9XsYOKGY4GHONfdHX6C83Q4WwGRcnE+90r8f/ejdk1+krdJ+wmYgLZtSAKdcIHvovRCyD3dOtH6idcIyEfmQpZGVAvSfMPuSvm6G95GaoEDb3VEhF5o5sSVpGFk98t5XlB87nfVCrsVC1g1FqN+641WO0B46R0A8vBv+qxm9tM9xIzeC3ved4vGF5uRkqRDHRpFJJlo5tS+2yPoyZtZt3lx7KfRGSyQQ9JxqTIBaNhMxCbIPnIOw/oacmQuRmqN3V7OGWpfuMm6H9W0jdFiGKk9K+7swZ0YphbaowdUskfX7YRvSVXOar+5aDx7+EmHBju8n7nFkJXSnVSSl1VCl1Qik1LofnayultimlUpVS/7R8mLk4vREy06D6I2YfMnvnWWqV8ZGboUIUQ67OJv7brS7fP9OUU5cS6fr1ZtYeyaViY71eUL83bPgEzu0tsjiLozwTulLKCZgAdAbqAv2UUnXvaBYPvAh8bvEI83J8Nbh6Q6VWZjX/62Zov9CKcjNUiGKsc4NyLHuxLUElPRg+PYxPVkSQfq+pjV0+A69AWDQK0lOKNtBixJweeihwQmt9SmudBswBetzaQGt9SWu9CyjEDrEFoDUc/8O4MeLsatYhN2+GNpWboUIUd5UDvFgwujX9W1Ri4oaT9J64jci4G3c39PSH7t9C7BFY90HRB1pMmJPQKwC3FjOOzn7M9i4dgevRUONRs5rfdjPUQ26GCmEP3F2c+KhXAyb0b0pk3A26fL2JubvO3l2Kt8bDEDIMtn4LkVtsE6yNmZPQcxqXKNBMfqXUCKVUmFIqLDY2tiCnuF3kJuPPah3Nav73zVBZGSqEvenasBwrX25Ho6AS/HvBAUbP3H33QqRH3oeSlWHx6PtyQwxzEno0cOt0kCDgXEFeTGs9SWsdorUOCQwMLMgpbndmK/gGGStEzfDLzZuhJQr/2kKIIlfOz4NZz7bg9c61WRtxkU7jN7Lx2C2dQzdvoyrj1bOw6k3bBWoj5iT0XUANpVQVpZQr0BdYYt2wzKA1nN0Olc2/Gbo/+hr9W8jKUCHsmcmkGNm+GovGtMHH3YVBU3YybsF+rqdk38Kr1BLavGisID222rbBFrE8E7rWOgMYC6wCjgDztNaHlFKjlFKjAJRSZZVS0cCrwFtKqWillK81A+fKaUi8YPzjmWHurijcnE30bFw8hv+FEIVTv4Ify15oy8j2VZkXFsWj/7eRdRGXjCc7vgmBdWDpS5ByzbaBFiGz5qFrrZdrrWtqratprT/Mfmyi1npi9vcXtNZBWmtfrXWJ7O/NKMhQCGe3G39Wap1n05T0TBbvjaFLAymTK4QjcXdx4vXOdVg0pg2+Hs4MnbaLV+ft5Wqagp4TjE7f6rdsHWaRsd+VojHh4OYLgbXzbLri4HkSUjJ4KkRWhgrhiBpVLMHSF9rywoPV+W3vOR75ciNL48qhW71gbHxz8k9bh1gk7DehXzgIZeoZ9RzyMHdXFJUDPGlZ1b8IAhNC2IKbsxP/eLQWvz3fhjK+brwwew/DzzxMWomqsOQlo0yIg7PPhK41XDxkJPQ8nLl8g+2n4nkqRFaGCnE/qF/Bj9+eb8u73euxKzqZQXGD0deiyFj9tq1Dszr7TOhXz0BaApSpn2fTeWFRmBQ8KStDhbhvOJkUg1sHs/af7SlTvz1TMx7DOfwndm9cdveCJAdinwn9wkHjzzwSekZmFr+GRdOhVmnK+rkXQWBCiOKktI874/s2oc6AzzinyuK/5lWe/WmjeZto2CH7TOiXs4vZl879huiGY7FcSkjl6eZyM1SI+1mr2pUIfGYSwaaLtI+ZRJfxm3h94QFiE1JtHZpF2WdCv3oWPPzBzSfXZnN3RVHK240Ha5cuosCEEMWVS/X2EDKMgfzOGw0T+TUsio6fr2fCuhMkp2XaOjyLsNOEHgUlcu91xyak8mfEJZ5sWgEXJ/v8awohLOyR91B+QTx7+QtWv9iCllUD+GzVUR74bB3Tt0aSmmHfid0+M93Vs3nWb1m4O5qMLE0fmXsuhPiLm4+xw1HcUapG/MRPg0OYN7IVVUp58faSQzz4+Qbm7Yoi415114s5+0voWsO1KPC7d0LXWjN3VxQhlUtSvbR3EQYnhCj2ajwC9Z+ETZ9D7DFCq/gzd0RLZgwLpZS3K/9asJ9HvtzIoj3RdpfY7S+hp92A9CTwKXPPJmFnrnAq7gZPyc1QIUROOn0CLh6w7GXIykIpxQM1A1n8fBsmDWyGm7OJV+buo+MX65m5/Qwp6fYxFGOfCR2MbefuYe6uKLzdnOnaoFwRBSWEsCvepY3a6We2wN6ZNx9WSvFovbIsf7EdkwY2w9/LjbcWH6Td/9bxw4aTJKZm2DDovNlhQs9evnuPhJ6Qks7v+8/TrVE5vNycizAwIYRdaTLQKO63+j+QeOm2p0wmI7EvHtOaX55tQa0yPny8IoLWH6/lkxURxFxNtlHQubPDhP5XD90rx6eX7T9PcnqmFOISQuTOZIJuXxlDuCtfz7GJUorW1Usx89kW/PZ8G9pUL8WkjSd54H/rGDMrnJ2n44vVylP768LmkdDn7oqiZhlvGlcsUXQxCSHsU2AtaPsqbPgEGvUz9iW9h0YVS/D9gGZEX0ni5+1nmLMziuUHLlC3nC9D2gTTrWF5PFydijD4u9lxD/3uIZejFxLYG3VVCnEJIczX7lUIqAG/v/J3fslFUElPXu9ch+2vP8THTzQgM0vzr/n7Cf1wDW8tPsCB6Gs267XbYULP3vg1hx763F1RuDgpnpBCXEIIczm7QbfxxvqW9R+bfZiHqxP9Qiux8uV2zB3RkkfqluHXsGi6fbuZrl9vZvrWSK4lpVsx8LspW/0mCQkJ0WFhYfk/8OpZOLMNanUG9793uUvNyKTlR2tpXa0UE55pasFIhRD3hSUvwJ5ZMGI9lGtYoFNcS05nyb5zzN11loMx13F1MtGxdiA9GlfgwdqlcXcp/JCMUipcax2S03P2N4ZeolKOq0TXHL7ElaR0mXsuhCiYR96DiOXw+z9g2CqzNs+5k5+HCwNbVmZgy8ocjLnGgt3RLNt/nlWHLuLt5sxj9crSo3F5WlcLwNkKJUnsb8jlHuaGRVHez5221UvZOhQhhD3yKGkk9eidt81NL6j6Ffx4u1s9tr/+ELOebUGXBmVZffgCg6bs5L1lhy0Q8N3sr4eeg+grSWw6HssLD9bAySQ3Q4UQBdSon7EH6R9vQ+3HwbPw21Y6mRRtqpeiTfVSvNejPuuPxlLR38MCwd7NIXro88OjAejTTG6GCiEKwWSCrl9AyjVY+67FT+/u4kSn+mWpV97P4ucGB0joWVmaX8OiaVOtFBX9PW0djhDC3pWtDy1GQfh0iA63dTT5YvcJfcvJOGKuJsuuREIIy+kwDrzLGHPTs+yjMBc4QEKfuyuKEp4uPFrv3tUXhRAiX9x94bEP4fw+CJti62jMZtcJ/cqNNFYfukjPxhVwc7btklshhIOp/yRUaQ9r37+reFdxZdcJfdGeGNIys2S4RQhheUoZN0jTk+CP/9o6GrPYbULXWjMvLIqGQX7UKeeb9wFCCJFfpWpA6xdg32yI3GLraPJktwl9f/Q1Ii4kSJlcIYR1PfAa+FWEFf+CTNngwirmhkXh7mKie+Pytg5FCOHIXD3h0Q/g4kHYPc3W0eTKLhN6UloGS/eeo0uDcvi6u9g6HCGEo6vbA4LbwZ8fQFK8raO5J7MSulKqk1LqqFLqhFJqXA7PK6XU19nP71dKWbXc4fIDF0hIzeBpGW4RQhQFpaDzp8YK0nUf2Tqae8ozoSulnIAJQGegLtBPKVX3jmadgRrZXyOA7y0c523m7YqiSikvQqsUvs6CEEKYpUw9aP4shE2GCwdtHU2OzOmhhwIntNantNZpwBygxx1tegAztGE7UEIpVc7CsQJwKjaRnZHx9AkJkl2JhBBFq8Pr4F4CVo6DYrSX6F/MSegVgKhbfo7Ofiy/bVBKjVBKhSmlwmJjY/MbKwAnLiXi7+VKb9mVSAhR1Dz94cE3IXITHP7N1tHcxZyEnlM3+M5fTea0QWs9SWsdorUOCQwMNCe+uzxaryw733iI0r7uBTpeCCEKpdlQKNMAVr8FaUm2juY25iT0aODWu49BwLkCtLEYa+z0IYQQZjE5GTdIr0XB1q9tHc1tzMmMu4AaSqkqSilXoC+w5I42S4BB2bNdWgLXtNbnLRyrEEIUD8FtoN4TsPlLY5/jYiLPhK61zgDGAquAI8A8rfUhpdQopdSo7GbLgVPACeBHYIyV4hVCiOLh0fcBBav/Y+tIbjJrCzqt9XKMpH3rYxNv+V4Dz1s2NCGEKMb8gqDtK7D+Izi9Caq0s3VE9rlSVAghioU2Lxp1Xla9AVlZto5GEroQQhSYiwc89DZc2A/759g6GknoQghRKPWfhArNjI0w0m7YNBRJ6EIIURgmEzz2ESScg63f2jYUm766EEI4gkotjYqMW76C67absS0JXQghLOHhdyArA9Z9YLMQJKELIYQl+FeF0BGwZxac32+TECShCyGEpTzwGniUhNVv2qQaoyR0IYSwFI8SRond0xvh2Koif3lJ6EIIYUkhQyGghlGNMTO9SF9aEroQQliSk4tR5+XycQibWqQvLQldCCEsrWYnqPIArP8Ykq8W2ctKQhdCCEtTCh79AJLjYcv4IntZSehCCGEN5RpBg6dg+/dw3Wr7/dxGEroQQljLg2+CzoR1HxXJy0lCF0IIaykZDM2fhb2z4FKE1V9OEroQQlhTu3+CqzesecfqLyUJXQghrMkrANq+DMdWwJmtVn0pSehCCGFtLUaDT3n4479WLQkgCV0IIazN1RM6vg7Ru+DIUqu9jCR0IYQoCo36Q2BtWPuu1UoCSEIXQoii4ORs1Ey/fAJ2z7DKS0hCF0KIolKzE9TvbZTYtQJnq5xVCCHE3ZSC3pOtdnrpoQshhIOQhC6EEA5CEroQQjgISehCCOEgJKELIYSDkIQuhBAOQhK6EEI4CEnoQgjhIJS2YuWvXF9YqVjgTAEPLwXEWTAcSymucUHxjU3iyh+JK38cMa7KWuvAnJ6wWUIvDKVUmNY6xNZx3Km4xgXFNzaJK38krvy53+KSIRchhHAQktCFEMJB2GtCn2TrAO6huMYFxTc2iSt/JK78ua/isssxdCGEEHez1x66EEKIO0hCF0IIB1HsErpSqpNS6qhS6oRSalwOzyul1NfZz+9XSjU191grx/VMdjz7lVJblVKNbnkuUil1QCm1VykVVsRxdVBKXct+7b1Kqf+ae6yV43rtlpgOKqUylVL+2c9Z8/2aopS6pJQ6eI/nbXV95RWXra6vvOKy1fWVV1xFfn0ppSoqpdYppY4opQ4ppV7KoY11ry+tdbH5ApyAk0BVwBXYB9S9o00XYAWggJbADnOPtXJcrYGS2d93/iuu7J8jgVI2er86AMsKcqw147qjfTfgT2u/X9nnfgBoChy8x/NFfn2ZGVeRX19mxlXk15c5cdni+gLKAU2zv/cBjhV1/ipuPfRQ4ITW+pTWOg2YA/S4o00PYIY2bAdKKKXKmXms1eLSWm/VWl/J/nE7EGSh1y5UXFY61tLn7gfMttBr50prvRGIz6WJLa6vPOOy0fVlzvt1LzZ9v+5QJNeX1vq81np39vcJwBGgwh3NrHp9FbeEXgGIuuXnaO5+Q+7VxpxjrRnXrYZj/Bb+iwZWK6XClVIjLBRTfuJqpZTap5RaoZSql89jrRkXSilPoBOw4JaHrfV+mcMW11d+FdX1Za6ivr7MZqvrSykVDDQBdtzxlFWvr+K2SbTK4bE751Xeq405xxaU2edWSnXE+A/X9paH22itzymlSgN/KKUisnsYRRHXbozaD4lKqS7AYqCGmcdaM66/dAO2aK1v7W1Z6/0yhy2uL7MV8fVlDltcX/lR5NeXUsob4xfIy1rr63c+ncMhFru+ilsPPRqoeMvPQcA5M9uYc6w140Ip1RD4Ceihtb781+Na63PZf14CFmF8vCqSuLTW17XWidnfLwdclFKlzDnWmnHdoi93fBy24vtlDltcX2axwfWVJxtdX/lRpNeXUsoFI5nP0lovzKGJda8vS98YKMwXxieGU0AV/r4xUO+ONl25/abCTnOPtXJclYATQOs7HvcCfG75fivQqQjjKsvfC8hCgbPZ751N36/sdn4Y46BeRfF+3fIawdz7Jl+RX19mxlXk15eZcRX59WVOXLa4vrL/3jOAr3JpY9Xry2JvrgX/kbpg3B0+CbyZ/dgoYNQtb9qE7OcPACG5HVuEcf0EXAH2Zn+FZT9eNfsfZx9wyAZxjc1+3X0YN9Na53ZsUcWV/fMQYM4dx1n7/ZoNnAfSMXpFw4vJ9ZVXXLa6vvKKy1bXV65x2eL6whgG08D+W/6duhTl9SVL/4UQwkEUtzF0IYQQBSQJXQghHIQkdCGEcBCS0IUQwkFIQhdCCAchCV0IIRyEJHQhhHAQ/w9aIdoxn8o3xQAAAABJRU5ErkJggg==\n",
       "text/plain": [
        "<AxesSubplot:>"
       ]
      },
      "metadata": {
       "needs_background": "light"
      },
      "output_type": "display_data"
     }
   ],
   "source": [
     "# The .figure() method will create a new figure, or activate an existing figure.\n",
    "plt.figure()\n",
    "# The .plot() is a versatile function, and will take an arbitrary number of arguments. For example, to plot x versus y.\n",
    "plt.plot(X, Y, label='actual')\n",
    "plt.plot(X, predict(Xf, W), label='predicted')\n",
    "# The .legend() method will place a legend on the axes.\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2021 Google Inc. Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
